Backslash u produces a parser error

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Backslash u produces a parser error

Jayet, Patrick
Hi all,

I am getting a (somehow weird) error with the groovy parser if there is a backslash u '\u' somewhere in a Groovy source file [0]. This can happen for instance if I paste as comment a path on windows, where a subfolder has a name starting with 'u' (e.g. \foo\url). In this case I get the stacktrace [1] as a warning at the top of the file.

Since the class cannot be parsed, references to it from other scripts or classes (e.g. in import statements) produce an error.

I am using Greclipse 2.1.1 on windows XP. The workspace encoding is set to CP1252.

Is this a bug in the Groovy parser?

Thanks in advance.
Cheers,

Patrick


[0] Example of a groovy script with the problem

// hello \u
class Foo{}


[1] Stacktrace

antlr.TokenStreamIOException: Did not find four digit hex character code. line: 3 col:9
 at org.codehaus.groovy.internal.antlr.parser.GroovyLexer.nextToken(GroovyLexer.java:710)
 at org.codehaus.groovy.internal.antlr.parser.GroovyLexer$1.nextToken(GroovyLexer.java:249)
 at antlr.TokenBuffer.fill(TokenBuffer.java:69)
 at antlr.TokenBuffer.LA(TokenBuffer.java:80)
 at antlr.LLkParser.LA(LLkParser.java:52)
 at org.codehaus.groovy.internal.antlr.parser.GroovyRecognizer.identifier(GroovyRecognizer.java:1740)
 at org.codehaus.groovy.internal.antlr.parser.GroovyRecognizer.packageDefinition(GroovyRecognizer.java:941)
 at org.codehaus.groovy.internal.antlr.parser.GroovyRecognizer.compilationUnit(GroovyRecognizer.java:686)
 at org.codehaus.groovy.antlr.ErrorRecoveredCSTParserPlugin.transformCSTIntoAST(ErrorRecoveredCSTParserPlugin.java:62)
 at org.codehaus.groovy.antlr.AntlrParserPlugin.parseCST(AntlrParserPlugin.java:79)
 at org.codehaus.groovy.control.SourceUnit.parse(SourceUnit.java:260)
 at org.codehaus.groovy.control.CompilationUnit$8.call(CompilationUnit.java:179)
 at org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:943)
 at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:574)
 at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:550)
 at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:527)
 at org.codehaus.jdt.groovy.internal.compiler.ast.GroovyCompilationUnitDeclaration.processToPhase(GroovyCompilationUnitDeclaration.java:165)
 at org.codehaus.jdt.groovy.internal.compiler.ast.GroovyParser.dietParse(GroovyParser.java:309)
 at org.codehaus.jdt.groovy.integration.internal.MultiplexingCommentRecorderParser.dietParse(MultiplexingCommentRecorderParser.java:60)
 at org.eclipse.jdt.internal.compiler.parser.SourceTypeConverter.convert(SourceTypeConverter.java:159)
 at org.eclipse.jdt.internal.compiler.parser.SourceTypeConverter.buildCompilationUnit(SourceTypeConverter.java:121)
 at org.eclipse.jdt.internal.core.hierarchy.HierarchyResolver.resolve(HierarchyResolver.java:698)
 at org.eclipse.jdt.internal.core.hierarchy.HierarchyResolver.resolve(HierarchyResolver.java:613)
 at org.eclipse.jdt.internal.core.hierarchy.HierarchyBuilder.buildSupertypes(HierarchyBuilder.java:125)
 at org.eclipse.jdt.internal.core.hierarchy.IndexBasedHierarchyBuilder.build(IndexBasedHierarchyBuilder.java:135)
 at org.eclipse.jdt.internal.core.hierarchy.TypeHierarchy.compute(TypeHierarchy.java:300)
 at org.eclipse.jdt.internal.core.hierarchy.TypeHierarchy.refresh(TypeHierarchy.java:1263)
 at org.eclipse.jdt.internal.core.CreateTypeHierarchyOperation.executeOperation(CreateTypeHierarchyOperation.java:90)
 at org.eclipse.jdt.internal.core.JavaModelOperation.run(JavaModelOperation.java:728)
 at org.eclipse.jdt.internal.core.JavaModelOperation.runOperation(JavaModelOperation.java:788)
 at org.eclipse.jdt.internal.core.SourceType.newSupertypeHierarchy(SourceType.java:726)
 at org.eclipse.jdt.internal.core.SourceType.newSupertypeHierarchy(SourceType.java:678)
 at org.eclipse.jdt.internal.junit.util.CoreTestSearchEngine.isTestImplementor(CoreTestSearchEngine.java:115)
 at org.eclipse.jdt.internal.junit.launcher.JUnit3TestFinder.isTestImplementor(JUnit3TestFinder.java:91)
 at org.eclipse.jdt.internal.junit.launcher.JUnit3TestFinder.isTest(JUnit3TestFinder.java:87)
 at org.eclipse.jdt.internal.junit.util.CoreTestSearchEngine.isTestOrTestSuite(CoreTestSearchEngine.java:54)
 at org.eclipse.jdt.internal.junit.JUnitPropertyTester.isJUnitTest(JUnitPropertyTester.java:113)
 at org.eclipse.jdt.internal.junit.JUnitPropertyTester.canLaunchAsJUnitTest(JUnitPropertyTester.java:88)
 at org.eclipse.jdt.internal.junit.JUnitPropertyTester.test(JUnitPropertyTester.java:71)
 at org.eclipse.core.internal.expressions.Property.test(Property.java:58)
 at org.eclipse.core.internal.expressions.TestExpression.evaluate(TestExpression.java:99)
 at org.eclipse.core.internal.expressions.CompositeExpression.evaluateAnd(CompositeExpression.java:53)
 at org.eclipse.core.internal.expressions.AdaptExpression.evaluate(AdaptExpression.java:91)
 at org.eclipse.core.internal.expressions.CompositeExpression.evaluateAnd(CompositeExpression.java:53)
 at org.eclipse.core.internal.expressions.IterateExpression.evaluate(IterateExpression.java:150)
 at org.eclipse.core.internal.expressions.CompositeExpression.evaluateAnd(CompositeExpression.java:53)
 at org.eclipse.core.internal.expressions.WithExpression.evaluate(WithExpression.java:72)
 at org.eclipse.core.internal.expressions.CompositeExpression.evaluateAnd(CompositeExpression.java:53)
 at org.eclipse.core.internal.expressions.EnablementExpression.evaluate(EnablementExpression.java:53)
 at org.eclipse.debug.internal.ui.launchConfigurations.LaunchShortcutExtension.evalEnablementExpression(LaunchShortcutExtension.java:287)
 at org.eclipse.debug.internal.ui.launchConfigurations.LaunchConfigurationManager.getApplicableConfigurationTypes(LaunchConfigurationManager
.java:744)
 at org.eclipse.debug.internal.ui.launchConfigurations.LaunchConfigurationManager.getApplicableLaunchConfigurations(LaunchConfigurationManag
er.java:780)
 at org.eclipse.debug.internal.ui.contextlaunching.LaunchingResourceManager.getParticipatingLaunchConfigurations(LaunchingResourceManager.ja
va:506)
 at org.eclipse.debug.internal.ui.contextlaunching.LaunchingResourceManager.getLabel(LaunchingResourceManager.java:320)
 at org.eclipse.debug.internal.ui.contextlaunching.LaunchingResourceManager.computeLabels(LaunchingResourceManager.java:241)
 at org.eclipse.debug.internal.ui.contextlaunching.LaunchingResourceManager$2.run(LaunchingResourceManager.java:138)
 at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)
---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Backslash u produces a parser error

Daniel Henrique Alves Lima
        Is \u alone a valid Java syntax? I think \uHHHH is a unicode escape:

http://java.sun.com/docs/books/jls/second_edition/html/lexical.doc.html#100850

        Try this:

// hello \\u

or

// \\foo\\url


        Best regards,

                Daniel.

Jayet, Patrick wrote:
> Hi all,
>
> (...)

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Backslash u produces a parser error

Daniel Henrique Alves Lima
http://groovy.codehaus.org/Strings+and+GString

"(...) The groovy parser supports the notation \uab12 (i.e. a leading
backslash and precisely four hex digits after the 'u' ).
This notation can be used in strings or anywhere in the program like the
Java parser does. (...)"

Best regards,

Daniel.

Daniel Henrique Alves Lima wrote:
>     Is \u alone a valid Java syntax? I think \uHHHH is a unicode escape:
>
> (...)

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

AW: [groovy-eclipse-plugin-user] Backslash u produces a parser error

Jayet, Patrick
Hi Daniel,

You are right, \uXXXX is the escape sequence for unicode characters.

If I create the following class in Java:

  public class Bar {

  // hello \u

  }


then I get a compile error 'Invalid unicode', '\u' being underline in red.

 

I would expect the same from the Groovy editor. Otherwise the problem is hard to diagnose when it happens. In a groovy script, there is no error shown, just a warning at the top of the file with the stacktrace I included in my previous e-mail.

 

Should I open a Jira for this?

 

Cheers,

Patrick

 

Reply | Threaded
Open this post in threaded view
|

Re: Backslash u produces a parser error

Andy Clement
The reason you aren't seeing a nice error is that alongside the error
text, groovyc throws a very ugly exception, almost a crash, and we
don't handle that very well.  Raise a jira and we'll cope with this
coming out of groovyc.  I must say I don't expect groovyc to produce a
stack trace when there is a syntax problem, but it does (if you run
that script on the command line, you get the same big stack trace) -
might be worth a groovyc jira too.

Andy

On 4 February 2011 02:50, Jayet, Patrick <[hidden email]> wrote:

> Hi Daniel,
>
> You are right, \uXXXX is the escape sequence for unicode characters.
>
> If I create the following class in Java:
>
>   public
>
> class Bar {
>
>   // hello \u
>
>   }
>
> then I get a compile error 'Invalid unicode', '\u' being underline in red.
>
>
>
> I would expect the same from the Groovy editor. Otherwise the problem is
> hard to diagnose when it happens. In a groovy script, there is no error
> shown, just a warning at the top of the file with the stacktrace I included
> in my previous e-mail.
>
>
>
> Should I open a Jira for this?
>
>
>
> Cheers,
>
> Patrick
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Backslash u produces a parser error

Andrew Eisenberg
Just a warning that even if the stack trace is fixed, this file will
not work very well.  See this:
http://jira.codehaus.org/browse/GROOVY-4378

Groovy does not handle unicode literals very well.  The source
locations for unicode characters are incorrect and that means that has
an affect on the groovy editor.

On Fri, Feb 4, 2011 at 8:03 AM, Andy Clement <[hidden email]> wrote:

> The reason you aren't seeing a nice error is that alongside the error
> text, groovyc throws a very ugly exception, almost a crash, and we
> don't handle that very well.  Raise a jira and we'll cope with this
> coming out of groovyc.  I must say I don't expect groovyc to produce a
> stack trace when there is a syntax problem, but it does (if you run
> that script on the command line, you get the same big stack trace) -
> might be worth a groovyc jira too.
>
> Andy
>
> On 4 February 2011 02:50, Jayet, Patrick <[hidden email]> wrote:
>> Hi Daniel,
>>
>> You are right, \uXXXX is the escape sequence for unicode characters.
>>
>> If I create the following class in Java:
>>
>>   public
>>
>> class Bar {
>>
>>   // hello \u
>>
>>   }
>>
>> then I get a compile error 'Invalid unicode', '\u' being underline in red.
>>
>>
>>
>> I would expect the same from the Groovy editor. Otherwise the problem is
>> hard to diagnose when it happens. In a groovy script, there is no error
>> shown, just a warning at the top of the file with the stacktrace I included
>> in my previous e-mail.
>>
>>
>>
>> Should I open a Jira for this?
>>
>>
>>
>> Cheers,
>>
>> Patrick
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

AW: [groovy-eclipse-plugin-user] Backslash u produces a parser error

Jayet, Patrick
Hi Andrew,

This open Groovy issue is indeed uncool. For us it is not that bad, since we normally don't use unicode literals directly. What happens is that users of our DSL might occasionnaly paste a windows path as comment containing a '\u'. If the Groovy editor would show an error in such a case, that would be enough for us.

Cheers,
Patrick
---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: AW: [groovy-eclipse-plugin-user] Backslash u produces a parser error

Alberto Castro (G)

Hi there,

I absolutely agree about 'uncool' Please fix this.
I ran into this problem as well,  using references to branch specs in a comment statement.......  took for ever to figure this out and change the comment.

When the compiler senses a comment,  it should ignore everthing in it.  I can see how a problem like this can manifest itself further down the line inside Eclipse and STS plugins.......  which is why it took me a while to see this.  I thought the greclipse plugin had a bug until it also showed up outside Eclipse.

I now make it a practice that whenever I encounter a strange error,  to exit Eclipse and run the project in a command shell to see if the same problem occurs.  Talk about paranoia!

On Feb 4, 2011 12:27 PM, "Jayet, Patrick" <[hidden email]> wrote:
> Hi Andrew,
>
> This open Groovy issue is indeed uncool. For us it is not that bad, since we normally don't use unicode literals directly. What happens is that users of our DSL might occasionnaly paste a windows path as comment containing a '\u'. If the Groovy editor would show an error in such a case, that would be enough for us.
>
> Cheers,
> Patrick
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
> http://xircles.codehaus.org/manage_email
>
>
Reply | Threaded
Open this post in threaded view
|

AW: AW: [groovy-eclipse-plugin-user] Backslash u produces a parser error

Jayet, Patrick

Hi all,

 

Alright, I have filed a bug report regarding this issue:

http://jira.codehaus.org/browse/GRECLIPSE-986

 

Cheers!

Patrick