Single Line Comments in JavaCC

[mjdousti]

Hi all,

To be able to skip single-line comments starting with # in JavaCC, I found the following snippet:

SKIP :
{
    <"#" (~["\n","\r"])* ("\n" | "\r" | "\r\n")>
}

This snippet works well except when the comment is placed on the last line of input file; because EOF cannot be matched with any of "\n" | "\r" | "\r\n". I tried to add <EOF> at the end of the regular expression but JavaCC gave me an error saying that it cannot identify <EOF>.

Does anyone know how to handle this problem?

Thanks.

[kjkrum]

Does JavaCC recognize '$' as the end of a line?

[pbrockway2]

Have you tried the construction mentioned in the Faq (3.15: Why do the example Java and C++ token managers report an error when the last line of a file is a single line comment?). It's written for // comments, but perhaps it applies to # comments as well where the file doesn't end in a new line.

[JosAH]

Have you tried the construction mentioned in the Faq (3.15: Why do the example Java and C++ token managers report an error when the last line of a file is a single line comment?). It's written for // comments, but perhaps it applies to # comments as well where the file doesn't end in a new line.

Yep, that's the solution; I once hacked it by fiddling with the InputStream, i.e. it would synthesize a last new line sequence if the eof was reached before an end of line was read; the solution you mentioned is much less hackerish. (simply replace the // by # in the example).

kind regards,

Jos

[pbrockway2]

I just got bored and googled... I've never used JavaCC.

They mention in that Faq that "some compilers are more liberal and do not insist on that final newline". Is that the case with Java? I mean, are .java files *supposed* to end with a newline (like .c ones)? If so, it's the first I've heard of that...

[JosAH]

I just got bored and googled... I've never used JavaCC.

They mention in that Faq that "some compilers are more liberal and do not insist on that final newline". Is that the case with Java? I mean, are .java files *supposed* to end with a newline (like .c ones)? If so, it's the first I've heard of that...

I checked the JLS: it mentions line comments as a sequence of chars following a // pair and ending with a new line sequence. That, to me, implies that a lexically correct Java sourse code file that has a one line comment on the last line should end with an end of line sequence. Note that the unicode escape sequence \u000a counts as an end of line sequence; this leaves the question: does a last line that doesn't contain a single line comment nees to have an end of line sequence? It must be somewhere in the JLS but I'm upgrading my Eclipse and Java installation at the moment. so if you're still bored, be my guest ;-)

kind regards,

Jos

[pbrockway2]

That, to me, implies that a lexically correct Java sourse code file that has a one line comment on the last line should end with an end of line sequence

True. A quick check with Eclipse suggests that its compiler is of the "liberal" variety.

But where does the JLS say that? I see 3.7 (Comments):
Comment: EndOfLineComment
EndOfLineCmment: / / CharactersInLine_opt
CharactersInLine:
InputCharacter
CharactersInLine InputCharacter

3.4 Line Terminators says,
InputCharacter: UnicodeInputCharacter but not CR or LF

That would suggest that a comment can consist of two / followed by a whole bunch of characters (not CR or LF) and so may validly end a program source file.

-----

18.1 (the Grammar) says
CompilationUnit: [[Annotations] package QualifiedIdentifier ; ] {ImportDeclaration} {TypeDeclaration}

ie some optional stuff, followed by zero or more declarations. Which again suggests a new line isn't required. (the CompilationUnit is the "goal symbol"). Indeed the compilation unit might be empty.

[JosAH]

Near the top of 3.7:

There are two kinds of comments:

/* text */ A traditional comment: all the
text from the ASCII characters /*
to the ASCII characters */ is ignored
(as in C and C++).

// text A end-of-line comment: all the text
from the ASCII characters // to
the end of the line is ignored (as in C++).

To me that says that an end of line should be present for a single line comment ...

kind regards,

Jos

[mjdousti]

A great discussion! Thanks to pbrockway2 and JosAH.

So, let me summarize the solution. It seems that JavaCC doesn't understand "$". Adding a "?" at the end of regular expression like below allows the lexical analyzer to skip the comment line till reaching a new line or even finishing the read without reading more characters, i.e. reaching end of file.

SKIP :
{
    <"#" (~["\n","\r"])* ("\n" | "\r" | "\r\n")[COLOR="red"][B]?[/B][/COLOR]>
}