<![CDATA[Java Programming Forum - Learn Java Programming - Blogs - JosAH's blog by JosAH]]> http://www.java-forums.org/blogs/josah/ Java Programming Forum - Learning Java easily en Sat, 25 May 2013 12:21:10 GMT vBulletin 60 http://www.java-forums.org/images/misc/rss.jpg <![CDATA[Java Programming Forum - Learn Java Programming - Blogs - JosAH's blog by JosAH]]> http://www.java-forums.org/blogs/josah/ The Decorator pattern http://www.java-forums.org/blogs/josah/80-decorator-pattern.html Sat, 23 Jul 2011 12:42:35 GMT Greetings, the previous blog article talked a bit about the Visitor design pattern. This article talks a bit about additional functionality that... Greetings,

the previous blog article talked a bit about the Visitor design pattern. This article talks a bit about additional functionality that is sometimes wanted, i.e. the functionality is optional. Assume there is a lot of optional functionality that people want.

This article discusses the Decorator (or 'Wrapper') pattern. For the sake of the example we'll use array manipulation. People always fiddle diddle with arrays, i.e. they copy values from one array to another, move elements around in arrays etc. etc. as if there were no Object Oriented programming.

Since the early days of Fortran people have been messing around with arrays and mutilating those poor processors that have to copy data over and over all the time.

Let's do something about it: The following interface is an abstraction of an array:

Java Code: 
public interface Sequence<T> {
	public int length();
	public T get(int idx);
	public void set(T, int idx);
}
A Sequence has a length and we can get and set elements in a Sequence; nothing spectacular. Note that this interface is a generic interface: the Sequence manipulates elements of type 'T'.

Let's encapsulate a simple array:

Java Code: 
public class ArraySequence<T> implements Sequence<T> {

	// the encapsulated array:
	private T[] array;

	// the constructor:
	public ArraySequence(T[] array) { this.array= array; }

	// interface implementation:
	public int length() { return array.length; }
	public T get(int idx) { return array[idx]; }
	public void set(T elem, int idx) { array[idx]= elem; }
}
Why build all that code for just a simple array? We could've used the array itself in the first place, so why all this complicated stuff? This is why: sometimes we want to manipulate arrays but we don't want to copy and move and swap all those elements over and over again: we build Decorators for it instead. The ArraySequence class does nothing special: it simply encapsulates an array for us. Let's build an abstract class from which we can easily build other concrete Sequence implementations:

Java Code: 
public abstract class AbstractSequence<T> implements Sequence<T> {
	// another sequence:
	protected Sequence<T> seq;

	// an abstract method that gives us an index value:
	protected abstract int index(int idx);

	// the constructor:
	public AbstractSequence(Sequence<T> seq) { this.seq= seq; }

	// interface implementation:
	public int length() { return seq.length(); }
	public T get(int idx) { return seq.get(index(idx)); }
	public void set(T elem, int idx) { seq.set(elem, index(idx)); }
}
The class by itself can't do anything (it's abstract), but it gives us all the functionality we need for concrete Sequence implementations. The following Sequence treats all elements of a Sequence in reverse order:

Java Code: 
public class ReverseSequence<T> extends AbstractSequence<T> {
	// implementation of index() method:
	protected int index(int idx) { return seq.length()-idx-1; }

	// the constructor:
	public ReverseSequence(Sequence<T> seq) { super(seq); }
}
You see no interface implementation in this class because everything already was implemented by the AbstractSequence; all that this class implements is the index() method which gives the additional functionality. Note that the encapsulated 'seq' Sequence is passed to the superclass in the constructor.

If you create a ReverseSequence, given another Sequence you can treat the other Sequence as if it were reversed (i.e. last element first and vice versa). The added functionality is implemented in the index() method. Here's a bit of code that shows how it's used:

Java Code: 
	Integer[] a= { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 }
	Sequence rev= new ReverseSequence<Integer>(
				new ArraySequence<Integer>(a));
	...
	for (int i= 0, n= rev.length(); i < n; i++)
		System.out.print(rev.get(i)+" ");
	System.out.println();
The output is: 9 8 7 6 5 4 3 2 1 0

From the snippet above you can see why a Decorator is also named a Wrapper: the Sequence objects 'wrap' other Sequence objects (passed to the constructors). Also note from the snippet above that our code didn't touch the array directly anymore after we've wrapped it in a Sequence.

Here's another Sequence implementation: it puts the elements from the first part of the other Sequence interleaved with the elements from the last part of the sequence:

Java Code: 
public class MergeSequence<T> extends AbstractSequence<T> {
	// implementation of index() method:
	protected int index(int idx) { 
		if ((idx&1) == 0) return idx/2; // idx is even
		return (seq.length()+idx)/2;    // idx is odd
	}

	// the constructor:
	public MergeSequence(Sequence<T> seq) { super(seq); }
}
If you use the code snippet above but use a MergeSequence instead of a ReverseSequence the output will be: 0 5 1 6 2 7 3 8 4 9. I leave it up to you what the output will be if you wrap your original array in both an MergeSequence and a ReverseSequence as in:

Java Code: 
Sequence magic= new MergeSequence<Integer>(
			new ReverseSequence<Integer>(
				new ArraySequence<Integer>(a)));
I also leave it up to you to figure out if it would make any difference if you'd used this wrapping order instead:

Java Code: 
Sequence magic= new ReverseSequence<Integer>(
			new MergeSequence<Integer>(
				new ArraySequence<Integer>(a)));
Note that if you wanted to do what these two Wrappers do using just a simple array you have to play clever tricks and/or use temporary array(s) to hold the intermediate results.

Here's a bit more complicated Sequence: it catenates two other sequences as if they were just one bigger Sequence. We do have some work to do in the implementation of the interface methods for this class so we don't extend from the AbstractSequence. Here goes:

Java Code: 
public class CatenateSequence<T> implements Sequence<T> {
	// the first and second Sequence:
	private Sequence<T> first, second;

	// little helper method:
	private boolean isFirst(int idx) { return idx < first.length(); }

	// implementation of index() method:
	private int index(int idx) { 
		if (isFirst(idx)) return idx;
		return idx-first.length();
	}
	
	// the constructor:
	public CatenateSequence(Sequence<T> first, Sequence<T> second) {
		this.first = first;
		this.second= second;
	}

	// interface implementation:
	public int length() { return first.length()+second.length(); }
	public T get(int idx) { 
		if (isFirst(idx)) return first.get(index(idx));
		return second.get(index(idx)); 
	}
	public void set(T elem, int idx) {
		if (isFirst(idx)) first.set(elem, index(idx));
		else second.set(elem, index(idx));
	}
}
The last piece of code shows how we can catenate three arrays together using the CatenateSequence Wrapper:

Java Code: 
Integer[] a= { 0, 1, 2 };
Integer[] b= { 3, 4, 5 };
Integer[] c= { 6, 7, 8, 9 };
...
Sequence cat= new CatenateSequence<Integer>(
			new ArraySequence<Integer>(a),
			new CatenateSequence<Integer>(
				new ArraySequence<Integer>(b),
				new ArraySequence<Integer>(c)));
If you print this Sequence, the output again will be: 0 1 2 3 4 5 6 7 8 9.

Note that sneakily I used another design pattern too for the CatenateSequence class: the Composite pattern. But we'll talk in a next tip about that one.

Guess what a mess we can make if we wrap the CatenateSequence in the other Sequences again. Note that not a single array element is moved or copied. We add functionality at will by wrapping up a Sequence in another Sequence that implements the functionality for us. We can wrap Sequences in any order we want and thus we can accomplish any functionality we want. We now have three classes:

1: ReverseSequence
2: MergeSequence
3: CatenateSequence

If we had implemented any or all of those functionalities using separate classes we would've ended up with a big number of classes. Now we only have three of them which we can combine in any order we want and as many as we want. (I've excluded the ArraySequence class in the count because we always need it to encapsulate raw, simple arrays).

The Readers, Writers, InputStreams and OutputStreams in the Java core classes are implemented in a similar way: you wrap (or 'decorate') byte or character streams using all these wrappers. Each wrapper adds a bit of functionality if you need it.

kind regards,

Jos ]]> JosAH http://www.java-forums.org/blogs/josah/80-decorator-pattern.html The Visitor pattern http://www.java-forums.org/blogs/josah/79-visitor-pattern.html Sat, 23 Jul 2011 12:35:54 GMT Greetings,

Java claims to support OO, so why not use it? In this article we're going to talk a bit about when and why to apply certain patterns. We'll start with the Visitor pattern. The pattern is also named 'double dispatch' which will become clear near the end of this little article. Here's the story:

Suppose some time ago you wrote a bunch of beautiful, efficient classes; they're real gems. For the sake of this article let's use the following classes:

Java Code: 
public class Beautiful { ... }
public class Efficient { ... }
public class Gem { ... }
There's no need to write more of those classes and there's no need to add more functionality to them; users always use your class objects in collections; they traverse the collections and call the fine functionality of the objects in those collections. An ideal solution.

Until one sad day a customer shows up, loaded with money and willing to pay and he wants some additional functionality added to your classes. He already uses your classes and he's very happy with them: he thinks they're very efficient, beautiful and real gems. but ... he wants a bit of functionality added to them. Let's call this customer Fred.

You do like the money Fred is willing to pay and you start adding the wanted functionality and you're doing quite fine until the next sad day Barney comes in at your office and he wants some additional functionality too. Above all he's willing to pay the money for it also. Happy to oblige you tell Barney that you're working on additional functionality for customer Fred too at that very moment and you're willing to accept the task of course.

Barney isn't charmed by the idea: he doesn't like Fred's added functionality in his version of the updated classes at all and he isn't willing to pay for it either. You decide that two different versions of your efficient, beautiful gems of classes isn't that bad and the money isn't bad either.

That night you've got a nightmare: Wilma, Betty, Pebbles, Bambam and even Dino enter your office and they all want some additional functionality added to your fine classes too. A maintenance nightmare rears its ugly head.

What to do? You don't want seven versions of your classes and maybe entire Bedrock wants additional functionality added to your classes, all different.

Then you wake up and find the solution: instead of implementing more and more versions of your fine classes you do this:

Java Code: 
public class Beautiful implements Visitee { 
	public void whoIsThere(Visitor v) { v.thisIsBeautiful(this); }
	// ...
}
public class Efficient implements Visitee { 
	public void whoIsThere(Visitor v) { v.thisIsEfficient(this); }
	// ...
}
public class Gem implements Visitee { 
	public void whoIsThere(Visitor v) { v.thisIsGem(this); }
	// ...
}
... and you add two simple interfaces:

Java Code: 
public interface Visitee {
	public void whoIsThere(Visitor v);
}
public interface Visitor {
	public void thisIsBeautiful(Beautiful b);
	public void thisIsEfficient(Efficient e);
	public void thisIsGem(Gem g);
}
Your class structure (Beautiful, Efficient and Gem) stays the same; that much you know and you only added one little method to each class and you've defined two interfaces. That's all you did.

Let's implement the functionality for customer Fred; you still have to implement it because you've accepted the money, but now you can implement the functionality in a separate class: the Fred class; you give it an Iterator over all the objects in Fred's collection and implement the functionality like this:

Java Code: 
public class Fred implements Visitor {
	private Iterator<Visitee> i;
	public Fred(Iterator<Visitee> i) { this.i= i; }

	public void especiallyForFred() {

		while (i.hasNex()) i.whoIsThere(this);
	}

	public void thisIsBeautiful(Beautiful b) {
		// added functionality for Beautiful
		// all specially made for Fred
	}

	public void thisIsEfficient(Efficient e) {
		// added functionality for Efficient
		// all specially made for Fred
	}

	public void thisIsGem(Gem g) {
		// added functionality for Gem
		// all specially made for Fred
	}
}
You tell customer Fred that all he has to do is to instantiate a Fred object with an Iterator over his collections and then call the especiallyForFred method. He gets his additional functionality then and there's no need for him (or his programmers) to do anything else.

I think you can imagine what Barney's class will look like. If Wilma, Betty, Bambam, Pebbles and even Dino show up you don't have to alter your fine classes; all you have to do is implement the functionality they want, all in a separate class, just as we did in the example above. Each customer has his own added functionality and neither functionality clashes with other functionalities (you simply don't pass them along to other customers). All that's in your maintenance list now are those separate Visitor classes, one for each customer).

This is what the Visitor pattern is all about: a fixed set of classes or interfaces all used in a fixed manner and additional functionality that has to be implemented for different situations. Your fixed set of classes will be the Visitees and the aditional functionalities will be implemented by the visitors. The modifications needed for the original fine classes are minor (just one little method added that implements the Visitee interface).

Of course the additional functionality still needs to be implemented but you've drawn that away from your original set of classes. And even more: you can add whatever additional functionality on demand without any modification to your original class ad nauseam. On the other hand: the Visitor pattern can only be applied when your set of classes is more or less fixed (you don't want to change those interfaces and previously existing Visitors again and again). But if your set of classes is fixed, go for it: implement Visitors in your existing code base and keep your classes clean. You just have to prepare your classes for Visitors that may not even have been designed and written yet, i.e. your fixed set of classes is prepared for an unknown future.

If you carefully read the example classes and interfaces above you'll notice that the Visitor starts by calling a Visitee; the Visitee responds by calling the appropriate method on the Visitor again. That's the reason why the Visitor pattern is dubbed the 'Double Dispatch' pattern: from Visitor to Visitee and back again. That's all that's needed to keep your classes clean and frees you from maintenance nightmares.

kind regards,

Jos ]]> JosAH http://www.java-forums.org/blogs/josah/79-visitor-pattern.html Compilers http://www.java-forums.org/blogs/josah/77-compilers.html Sat, 16 Jul 2011 14:41:45 GMT Greetings, Frequently a question pops up in this forum (or other forums for that matter) where the OP wants to implement a calculator or wants to... Greetings,

Frequently a question pops up in this forum (or other forums for that matter) where the OP wants to implement a calculator or wants to compile an expression given in text (String) form. Often they get lost in tokenizers or regular expressions or whatever. This article describes how a compiler (for expressions) can be designed and implemented. The attachment contains a zip file with all the sources for a complete compiler.

Top level architecture

A compiler consists of the following parts:

1) a lexical analyzer; this component chops up the character input stream and forms 'tokens'. e.g. the expression "x+4.2" contains three tokens: [x], [+] and [4.2]; the three tokens have the followng types: NAME, OPERATOR and DOUBLE. A lexical analyzer needs a Reader to read characters from. The stuff between square brackets are a textual representation of the tokens, not of the text the tokens are generated from.

2) a parser: this component works in lock step with a lexical analyzer, i.e. it asks the analyzer to produce tokens and the parser checks for the syntactic validity of the sequence of tokens. e.g. the text "*x-/y" produces the token sequence[*], [x], [-], [/] and [y]. Everything is fine according to the lexical analyzer but the parser protests because the original string doesn't make up a syntactically correct expression.

3) a backend or processor; this backend component is called by the parser to generate code or interpret the token stream generated by the lexical analyzer and checked by the parser. The zip file contains three different processor implementations: an interpreter, a tree generator and a code generator.

4) an evaluator: a processor delivers an evaluator that can produce the result of the expression. For an interpreter the result of the expression is already known and the evaluator just hands it back to the user; for a code generator, an evaluator runs the produced code and finally hands the result back to the user.

5) a compiler: this component is a 'facade' in pattern language; it generates the lexical analyzer, the parser and constructs the entire 'compiler' out of it. The user hands a processor to the compiler (used by the parser) so the entire engine can do what it has to do: produce a result to be handed back to the user.

As far as the user is concerned only 5) is of interest; it handles all the other components mentioned in 1) ... 4). The following parts describe the components mentioned in the five parts. The parser is of interest syntactically, i.e. it implements a 'recursive descent' parser for ordinary infix notation expressions.

The compiler conducts the following flow of data and actions:

reader --> analyzer --> parser --> processor --> evaluator.

The analyzer is only used by the parser and the processor is activated by that same parser. The parser is started by the compiler and the evaluator is generated by the processor which is also called by the compiler.

This all looks like this is going to be a big, fat bunch of code; while writing a compiler isn't a trivial task by far, it isn't rocket science either; even more important, the approach outlined in this artice works and all the struggling by those posters, juggling with regular expressions, tokenizers and semi-smart shortcuts doesn't. The scenario above is implemented by five classes that take at most two pages of source code each (including a lot of empty lines). To be fair, a few small utility classes are needed but they don't take up a single page of source code each. Another benefit of this scenario is that, say, a parser can be altered while the other components can stay as they are. The same is true for the processor component, e.g. the attachment contains three entirely different processor implementations demonstrating this independency.

The rest of this article briefly describes the (small) utility classes and the five components mentioned above that make up the compiler.

The utility classes

Token

An analyzer produces tokens given a character input stream. The Token class implements those tokens. A Token has three parts:

1) the type of the Token;
2) an associated value;
3) its textual representation.

The types of a Token are implemented as an enumeration. The types are:

Java Code: 
- EOF      : the end of the character input stream has been reached before the analyzer could produce any other Token;
- ERROR    : an error occurred while reading the character input stream; 
- INT      : an int literal number was read;
- DOUBLE   : a floating point literal number was read;
- CHAR     : any non specified character was read;
- OPERATOR : an operator was read;
- NAME     : a name was read;
- FUNC     : a function name was read.
For example, if the input stream contains the text "x=(3+4)" the following token types are produced: NAME, OPERATOR, CHAR, INT, OPERATOR, INT, CHAR, EOF. For the
token types INT, DOUBLE and OPERATOR the associated value contains an Int, a Double or an Obcode (see below) object. For all other types of Tokens the associated value equals null.

Opcode

This enumeration is used for all the operators in an expression. The values of the Opcode enumeration are:

Java Code: 
	OUT("out", -1),
	
	SOBINARY("",0),
	SEQ(","	,	0),
	MOV("="	,	1),
	EQL("==",	2),
	NEQ("!=",	2),
	LES("<",	3),
	LEQ("<=",	3),
	GEQ(">=",	3),
	GRT(">",	3),
	SHL("<<",	4),
	SHR(">>",	4),
	OR("|",	5),
	XOR("^",	5),
	AND("&",	6),
	ADD("+",	7),
	SUB("-",	7),
	MUL("*",	8),
	DIV("/",	9),
	MOD("%",	9),
	EOBINARY("",10),
	
	SOUNARY("",10),
	PLS("+",   10),
	MIN("-",   10),
	NOT("!",   10),
	INV("~",   10),
	INC("++",  10),
	DEC("--",  10),
	EOUNARY("",11),

	SOFUNC("", 11),
	SIN("sin", 11),
	COS("cos", 11),
	TAN("tan", 11),
	MNF("min", 11),
	MXF("max", 11),
	EOFUNC("", 12);
The SOBINARY, EOBINARY, SOUNARY, EOUNARY, SOFUNC and EOFUNC values mark the start and end of the binary operators, the start and end of the unary operators and the start and end of the functions (SO is 'Start Of' and 'EO' is 'End Of'). The OUT value is a special 'operator' that moves the result of the expression to a well known place. Note that functions and operators are considered similar; and they are: operators can be interpreted as functions taking one or two arguments and producing a result. Lisp is a fine example of this similarity, i.e. (+ 3 4) is considered to be a function + applied to its operators 3 and 4; the function value is 7 here (surprise!). In this example implementation five functions are implemented: sin, cos, tan, min and max. Each value has an associated precedence value: functions have the highest precedence, followed by the unary operators and finally the lowest precedence operators are the binary operators. Because we want to use ordinary infix notation we have to distinguish between operators and functions.

Symbols

This utility class helps the lexical analyzer, i.e. given a textual representation of an operator or function it can produce the corresponding Opcode (see previous paragraph). This class can also produce the 'arity' of an operator or function, i.e. it knows how many arguments the operator or function needs. It also knows which characters are used for parenthesized expression (the '(' and ')' characters) and it knows which character is used as a separator used in parameter lists (here the ',' is used (as in min(3, 4)).

Analyzer

The Analyzer object has to produce Token objects given a character input stream. Basically, what it does is this:

Java Code: 
			for (int c= read(); c != -1; c= read()) {
				
				if (Character.isWhitespace(c)) continue;

				if (Character.isDigit(c)) return numberToken();
				
				if (Character.isJavaIdentifierStart(c)) return nameToken(c);
				
				return tokenToken(c);
			}

			return new Token(Token.TokType.EOF, null, "eof");
It keeps reading characters and decides what to do given the fresh character:

- white space is simply consumed and skiped;
- a digit must be the start of a number;
- a character that is a Character.isJavaIdentifierStart(int) must be a name or a function name;
- otherwise it must be the start of an operator or just an ordinary character;
- if nothing can be read anymore an EOF token is produced.

The numberToken method uses a regular expression to match a Double type number or an Integer type number and produces a corresponding Token object. The nameToken method checks the characters to be the start of a Java identifier or part of a Java identifier. If the entire name has been read it checks whether or not the name represents a function and a corresponding Token object is produced. The tokenToken method reads the character input stream as long as the characters represent an operator. This scenario produces a single Token object for the character sequence ">>" instead of two Token objects [>] and [>]. This scenario also has the restriction that a character secuence p c can only be an operator if only p is a possible start of an operator; so @> can't be an operator because @ isn't an operator. The Analyzer always consumes the longest possible operator, i.e. "+++++x" produces the Tokens [++] [++] [+] [x] instead of [+] [+] [+] [+] [+] [x].

Finally the Analyzer implements a nice utility method: if the Parser detects a syntactic error, e.g. in the expression "x+3(a-4)" the left parenthesis wasn't expected (a '*' character is missing for instance) I want to produce this:

Java Code: 
x+3(a-4)
   ^

the utility method procuces the line with the single caret '^' character at the right position. For further details of the methods mentioned above see the source code in the zip file.

Parser

Many consider this object the central part of a compiler and in a certain way it is: it sucks Tokens out of the Analyzer and calls a Producer whenever something needs to be produced but all it does is check for a correct syntax of a Token stream. This Parser object implements 'normal' infix expressions (note the quotes because it is far from normal if your used to prefix notation of (god forbid) postfix notation). Some languages (FORTH comes to mind) don't even have a syntax, i.e. any sequence of Tokens makes up a valid sequence, which is nice imho, but our language does have syntax: infix expression syntax, made up by those math folks. On a top level an expression looks like this:

<expression> [operator] <expression> [operator] <expression> [operator] ...

where <expression> is any expression containing only operators (if any) with higher precedence than <operator>. On the top level none of the parts need to be present (an empty expression that doesn't do anything) or just the first part needs to be present (no operators with lowest precedence present in the expression). When we 'zoom in' one level we see the same thing again:

<expression> [operator] <expression> [operator] <expression> [operator] ...

where [operator] isn't an operator of the lowest precendence nor one with a precedence one level up. Here's an example: the expression:

Java Code: 
2*(3+4)
can be seen at the lowest level as: <expression>. One 'zoom in' step further, this <expression> can be seen as: <expression>*<expression>. The second <expression> can be seen as (<expression>) and this one can be interpreted again as <expression>+<expression>. Each 'zoom in' step only considers operators of higher level than at one 'zoom out' step.
Finally we have zoomed in that far that no binary operators are present anymore. We have reached the unary expressions, which are:

<unary operator>* <atomic expression>

where <unary operator> is one of +, -, ~ or ! and the <atomic expression> can be either a parenthesized expression, a numerical constant, a name or a function call. This is exactly what our parser does: zooming in recursively and calling the Processor object whenever it must be called (see the next pargraph). Whenever something goes wrong, i.e. the Parser doesn't 'see' whatever it wants to see during the recursive process, a syntax error is generated and the parsing process stops. See the attachment for the source code of the Parser class and its gory details described above.

In Backus Naur form this is the grammar accepted by the Parser:

Java Code: 
expression          : sequenceexpression { , sequenceexpression } *
sequenceexpression  : equalityexpression { [ == != ] equalityexpression } *
equalityexpression  : comparisonexpression { [ <  >  <= >= ] comparisonexpression } *
comparisonexpression: shiftexpression { [ >> << ] shiftexpression ) *
shiftexpression     : orexpression { [ | ^ ] orexpression } *
orexpression        : andexpression { & andexpression } *
andexpression       : addexpression { [ + - ] addexpression } *
addexpression       : unaryexpression { [* / % ] unaryexppression } *
unaryexpression     : [ + - ~ ! ] unaryexpression | atomicexpression
atomicexpression    : [ { ( expression ) }  DOUBLE INT name function ( { sequenceexpression { , sequenceexpression } * }? ) ]
Anything between [ ... ] denotes a choice; anything between { ... } is just grouping and the * symbol denotes zero or more times of the previous and the ? symbol denotes zero or one times of the previous. All except the last two rules represent the 'zooming' decribed above and it's implemented by just three highly recursive methods. Note that the parameter list of a function call doesn't accept expressions but only sequenceexpressions; expressions can contain commas (,) which are also used to separate the individual parameters. Parenthesizing circumvents this slight inconvenience, e.g min(2, (1, 3)) is identical to min(2, 3) and results in the value 2.

Processor

The Processor is an interface; three implementations are present: Interpreter, TreeProcessor and CodeGenerator. The first implementation evaluates the expression while it is being parsed and scanned while the third implementation generates code for a (tiny) virtual machine. The TreeGenerator can generate a tree given a infix expression and produce three of its corresponding text forms: infix, postfix and prefix expressions. See the source code of the *Test.java files for details. Here is the Processor interface:

Java Code: 
package compiler;

public interface Processor {

	public static class ProcessorException extends RuntimeException {

		private static final long serialVersionUID = 5131994858045876960L;

		public ProcessorException(String message) {
			super(message);
		}
	};
	
	public Name setName(String name, Number value);
	
	public Evaluator getEvaluator();
	
	public void processBinary(Token token);
	public void processUnary(Token token);
	public void processDouble(Token token);
	public void processInt(Token token);
	public void processName(Token token);	
	public void processFunction(Token token);
}
It defines a (nested) Exception to be used by implementations of the Processor interface in case anything goes wrong. It is generated whenever an expression wants to assign to or modify anything but a name; e.g "54= 42" or "++42". While both these expressions are syntactically correct (the assignment is a binary operator syntactically speaking) they are nonsense semantically speaking. A Parser doesn't know anything about semantics, it just checks the syntax of a stream of Tokens. The setName( ... ) method can be used by a user to preset a Name to a certain value and the getEvaluator() method retrieves an Evaluator that can produce the result of the compiled expression.

The rest of the methods can be called by a Parser object when it has parsed a binary expression, a unary expression, a double type constant, in int type constant, a name or a function respectively. If the expression was "3*(4.5+x)" the following methods are called in that particular order:

Java Code: 
1) processInt(3)
2) processDOuble(4.5)
3) processName(x)
4) processBinary(+)
5) processBinary(*)
Interpreter

An Interpreter implements the Processor interface and is a simple stack machine. A stack can store Number type objects and an interpreter just does what it has to do when one of its methods are called. For the expression "x=2, 3*(4.5+x)" the sequence:

Java Code: 
processName(x)
processInt(3)
processBinary(=)
processDouble(4.5)
processName(x)
processBinary(+)
processBinary(*)
processBinary(,)
causes the following push and pop actions; the pop actions are mostly commented between parentheses:

Java Code: 
push x (a new name with the value set to 0)
push 2
pop (2)
pop (x the variable)
push 2.0 (the new value of x)
push 3
push 4.5 
push x (which is 2)
pop (2 the variable x value)
pop 4.5
push 6.5 (after addition)
pop (6.5)
pop (3)
push 19.5 (afer multiplication)
pop (19.5)
pop (2.0)
push 19.5
pop (19.5 the result of the entire expression)

An Interpreter object can be constructed with a boolean parameter; if it is set to true the entire evaluation is printed on the standard output stream which can be handy for testing purposes or testing. If the parameter value is set to false nothing is printed.

TreeProcessor

This class generates an AST (Abstract Syntax Tree) and can produce three of its equivalent text forms: a prefix, an infix and a postfix form. This Processor doesn't have an associated Evaluator, i.e. it does everything itself. Here is its output when the above expression is compiled:

Java Code: 
infix  : ((x = 2) , (3 * (4.5 + x)))
postfix: x 2 = 3 4.5 x + * ,
prefix:  (, (= x 2) (* 3 (+ 4.5 x)))
The TreeProcessor creates the AST using just two types of Nodes, i.e. an AtomicNode (for numbers and names) and a FunctionNode for everything else. Just because the infix notation isn't very consequent it distinguishes in the representation of operators and function, e.g. a binary operator is represented as: left operator right, while a binary function is represented as: operator(left, righ); but theyb behave identical if evaluated.

CodeGenerator

This class is also an implementation of the Processor interface but it generates instructions for a very simple virtual machine. This virtual machine is impemented by the CodeEvAluator class. When it is asked to produce the result of the compiled expression it runs the code and retrieves the result. The programming model for this tiny virtual machine is extremely simple: there are two spaces: a linear addressable data space that can contain Numbers (it's just an array of type Number) and code space which is just a List<Instruction>; an Instruction is also very simple: it is a so called 'three address instruction' where each address is an address in the data space (here it is an int index) and an opcode that tells what to do with the three addresses (and the Numbers in the data space). The instructions are on of:

Java Code: 
- data[rd]= data[rl] op data[rr]
- data[rd]= op data[rl]
- data[rd]= op
where rd, rl and rr are the three addresses and op represents what to do with the addresses (and the corresponding elements in the data space); note that the 'op' is implemented by the Opcode enum (see above). There is one instruction of the third form and it produces the desired output or result of the entire expression. For example, the expression "x=2, 3*(4.5+x)" produces the folling virtual machine code after execution:

Java Code: 
data:
[2.0, 2.0, 3.0, 4.5, 19.5]
code:
r0= r0 = r1
r4= r3 + r0
r4= r2 * r4
r4= out
the data space contains the items x, 2.0, 3.0, 4.5 and the value of the result. Note that variable x has the value 2.0 and the instruction sequence produces the value 19.5 in data[4]. The CodeGenerator object can be instantiated with a boolean parameter. If true is passed the code generator optimizes the code to be generated a bit, i.e. for the expression "+-+-+x" the instruction for x is generated, i.e. no code is generated for the unary plus operator and an even number of unary minus operators cancel out. The same holds for the expression "~~~~x", i.e. an even number of ~ operators cancel out as well. Also code for the , operator (the sequence operator) doesn't yield an instruction. If the parameter is false all spurious instructions are generated.

Concluding remarks

I realize that a lot of aspects and techniques aren't discussed in this little article; I leave that to the actual source code and the reader of it. The parsing part of it all uses old, very well established techniques: recursive descent parsing; the analyzer part isn't much miraculous either. The code generator part is new and self invented. It can be enhanced in numerous ways. The grammer being parsed by the parser can also be enhanced so the parser can parse much more than just infix mathematical expressions. Maybe other datatypes can be included (Strings?). The Interpreter uses techniques that can be found in many text books: it's a simple stack machine.

Both the Interpreter as well as the CodeGenerator (and its accompanying CodeEValuator) evaluate all numbers as if they were of type Double; I left the distinction in (generated by the Analyzer) just in case anybody feels tempted to implement two versions of all operators, one for the Integer operands and one for the Double type operands. I left the distinction out of the Processor implementations for reasons of simplicity.

Three small test programs are present in the zip file: InterpreterTest, TreeProcessorTest and GeneratorTest that test the Interpreter, the TreeProcessor and CodeGenerator Processor implementations a bit; play with them and do with the entire bunch whatever you want. Please mention my name in the source code (whether modified or not), especially if you want to make money with it ;-) If you ever see a question about expression evaluation pop up in one of the forums again, refer them to this blog article and don't think about it anymore.

kind regards,

Jos
Attached Files
]]> JosAH http://www.java-forums.org/blogs/josah/77-compilers.html Sudoku http://www.java-forums.org/blogs/josah/75-sudoku.html Thu, 23 Jun 2011 11:32:40 GMT Greetings, a couple of years ago a large part of the world went totally mad. Not because of global climate changes, not because of terrible wars... Greetings,

a couple of years ago a large part of the world went totally mad. Not because of global climate changes, not because of terrible wars that were started in the Middle East, nor because of global famine, nor because of large investment banks going bankrupt and nor because of Tsunamis, but because of a puzzle: Sudoku.

This is what Sudoku is all about:

Java Code: 
	+-------+-------+-------+ 
	| . . . | . . . | . . . | 
	| . . . | . . . | . . . |
	| . . . | . . . | . . . |
	+-------+-------+-------+ 
	| . . . | . . . | . . . |
	| . . . | . . . | . . . |
	| . . . | . . . | . . . |
	+-------+-------+-------+ 
	| . . . | . . . | . . . |
	| . . . | . . . | . . . |
	| . . . | . . . | . . . |
	+-------+-------+-------+
At the positions of the dots we're allowed to jot down a digit 1 ... 9 with the restriction that a digit only occurs exactly once in every row, column and in every little 3x3 sub-square. Sounds easy enough, but still millions of people all over the globe found this little puzzle much more important than the disasters described above.

Basic functionality

The first part of this article goes through the basic functionality needed by our Sudoku solver. I don't like these types of puzzles: I have to think a lot, jot down what I
did before, correct my mistakes etc. etc. So I decided to build a little program that would solve the thing for me. I took my scribbling paper and did this:

Java Code: 
             0 1 2   3 4 5   6 7 8
	  +-------+-------+-------+ 
	0 | . . . | . . . | . . . | 
	1 | . 0 . | . 1 . | . 2 . |
	2 | . . . | . . . | . . . |
	  +-------+-------+-------+ 
	3 | . . . | . . . | . . . |
	4 | . 3 . | . 4 . | . 5 . |
	5 | . . . | . . . | . . . |
	  +-------+-------+-------+ 
	6 | . . . | . . . | . . . |
	7 | . 6 . | . 7 . | . 8 . |
	8 | . . . | . . . | . . . |
	  +-------+-------+-------+
I merely numbered the rows, columns and I gave the nine little squares an index number too. I also decided that I wanted to fill every position with the digits 1 ... 9 but Java prefers index values 0 ... 8 instead.

Given a row index 'i' and a column index 'j' I want to check whether or not I can put a digit value 'val' in there. For the rows I need a boolean value that tells me if the value 'val' is already taken in row 'i':

Java Code: 
boolean[][] rows= new boolean[9][9];
...
if (row[i][val]) // 'val' is already present in row 'i'
The same for a value 'val' in column 'j':

Java Code: 
boolean[][] columns= new boolean[9][9];
...
if (columns[j][val]) // 'val' is already present in column 'j'
And then there are those little squares ... after a bit of scribbling I noticed the regularity and jotted down this:

Java Code: 
boolean[][] squares= new boolean[9][9];
...
if (squares[3*(i/3)+j/3][val]) // 'val' is already present in this square
And of course I need that little board itself:

Java Code: 
private int[][] board= new int[9][9];
So far so good: I can mark those three two dimensional arrays when I want to put a value 'val' there. Let's write some code for it; the 'val' value has to be a value in the range 0 ... 8 but I want to store values 1 ... 9 instead; here goes:

Java Code: 
private boolean possible(int i, int j, int val) {
		
	// position already taken or invalid?
	if (rows[i][val] || columns[j][val] || 
	    squares[3*(i/3)+j/3][val])
		return false;

	// position i,j is taken now:		
	rows[i][val]= true;
	columns[j][val]= true;
	squares[3*(i/3)+j/3][val]= true;
		
	board[i][j]= val+1;
		
	return true;
}
This little method returns false if the digit 'val' cannot be put at location i,j and it returns true if it can be stored there.

I want to remove a digit from the board too so I wrote this:

Java Code: 
private void reset(int i, int j) {
		
	// adjust to the range 0 ... 8
	int val= board[i][j]-1;

	// location i,j is free:		
	squares[3*(i/3)+j/3][val]= false;
	columns[j][val]= false;
	rows[i][val]= false;
}
Most (if not all of) those Sudoko puzzles have a few cells filled in already. Here's a convenient method that fills a cell with a digit:

Java Code: 
public boolean setValue(int i, int j, int val) {
		
	return possible(i, j, val-1);
}
This method assumes digits in the normal range 1 ... 9 and takes care of the adjustment to the range 0 ... 8. If the digit can not be set at that position for one reason or another, the method returns false. If there were no problems the method returns true.

And here's a simple method that gets a value at a position i,j:
Java Code: 
public int getValue(int i, int j) {
	return board[i][j];
}
We have the primitive functionality now: we can correctly get and set a cell value and we can reset it again. We can't solve the Sudoku puzzle yet nor can we conveniently set up the entire board and we can't even print it properly. Reading and writing entire Sudoku boards will be the subject of the second part of this article.

Sudoku I/O

The second part of the article defines two groups of methods. One method that is able to read an entire Sudoku board and a couple of methods that can write such a board, using a nice format.

We want a method that can read from a Reader stream and initialize the board accordingly. It would be very convenient if we could read a text file with content as shown in the example board as shown in th first part of this article.

Basically we want to read 81 digits or dots; a dot and a '0' both describe an empty cell, while the digits 1 ... 9 describe a filled cell. Here goes:

Java Code: 
public boolean read(Reader r) {
	
	int i= 0, j= 0; // the first position of the board
		
	try {
		// keep on reading characters:
		for (int x; (x= r.read()) != -1; ) {
			// skip it if not a digit nor a dot
			if (!(Character.isDigit(x) || x == '.')) continue;

			// is it a digit 1 ... 9?
			if (!(x == '0' || x == '.'))
					setValue(i, j, x-'0');

			// position i,j at next position, return when done
			if ((j= (j+1)%9) == 0)
				if (++i == 9) return true;
		}
	}
	catch (IOException ioe) { }
		
	// something went wrong
	return false;
}
The next group of methods can print a board given a Writer. It prints the board using the same format as shown in the example board shown in the first part of the article:

Java Code: 
 
// print a horizontal separator line
private void printHorizontal(PrintWriter p) {
	p.println("+-------+-------+-------+");
}

// print a little vertical line
private void printVertical(PrintWriter p) {
	p.print("| ");
}
	
// print the Sudoku board
public void print(Writer w) {
		
	PrintWriter p= new PrintWriter(w);
		
	for (int i= 0; i < rows.length; i++) {
		if (i%3 == 0) printHorizontal(p);
		for (int j= 0; j < columns.length; j++) {
			if (j%3 == 0) printVertical(p);
			p.print(board[i][j]+" ");
		}
		printVertical(p);
		p.println();
	}
	printHorizontal(p);
	p.flush();
}
Note that both the read method and the write method do not close the character streams; the Reader and Writer were passed in as a parameter so the caller of these methods is responsible for closing the streams. The read method does catch IOExceptions and simply returns false; all that the caller knows is that a board could not be read for some reason, i.e. an IOException was thrown or the content of the Reader didn't make up a valid board configuration.

Now we have all the primitive methods to manipulate a Sudoku board: we can test and set a value anywhere on the board, we can reset a cell again, we can initialize and entire board given a Reader and finally we can print the Sudoku board given a Writer.

The third and last part of the article shows the actual Sudoku solver. All methods shown above are part of a 'Sudoku' class. The solver method will also
be a member method.

Sudoku solver

This part describes the actual Sudoku solver. The solver works quite simple: given a cell to be filled in, try all possible values and solve the rest of the board. If all cells are filled the Sudoku puzzle is solved.

The solver iterates over all cells, left to rigth, top to bottom. It receives two parameters i and j which are the indexes of the cell to be filled in. The cell could be filled in already (it was a given cell value) in which case the solver has to find a next position. The first part of the solve method tries to find an empty cell:

Java Code: 
private boolean solve(int i, int j) {
		
	for(;;) {
		if (j >= columns.length) {
			j= 0;
			i++;
		}
		
		if (i >= rows.length) return true;
		
		if (board[i][j] > 0) j++;
		else break;
	}
	...
If you carefully inspect this piece of code you see that every row is checked, column by column. When the last column has been checked a next row is checked again until all cells are checked in which case the Sudoku puzzle is solved and true is returned. Otherwise an empty cell has been found and the piece of the code following this for loop is executed. The next piece of this method recursively tries to solve the Sudoku puzzle:

Java Code: 
	...
	for (int val= 0; val < squares.length; val++) {
		if (possible(i, j, val)) 
			if (!solve(i, j+1)) {
				reset(i, j);
			}
			else
				return true;
	}
	
	return false;
}
For the current cell all values 0 ... 8 are tried. If a value is feasible it is filled in and an attempt is made to solve the rest of the Sudoku puzzle. If the attempt fails the cell is reset and a next value is tried. If the attempt was successful the method returns immediately because the entire puzzle was solved. Otherwise the loop ends (none of the values were feasible for that particular cell i,j) and the method returns false.

Note that this method is a private method. I did that on purpose because I don't want other objects to supply the two index values. Here's a public method that does it for them:

Java Code: 
public boolean solve() {
	return solve(0, 0);
}
All we need now is a simple driver method that creates a Sudoku solver object, opens a Reader, initializes the puzzle and fires up the solver. We'll implement the driver functionality in the main() method for reasons of simplicity:

Java Code: 
 
public static void main(String[] args) throws Exception {

	Sudoku s= new Sudoku();
	FileReader fr= new FileReader(args[0]);

	if (s.read(new FileReader(args[0))) {
		if (s.solve()) {
			s.print(new OutputStreamWriter(System.out));
		}
		else
			System.err.println("problem cannot be solved");
	}
	else
		System.err.println("problem file cannot be read");
}
Note that this is a very sloppy implementation, i.e. when something goes seriously wrong, e.g. no file name was passed to the main method or the actual reading failed, an Exception is simply passed on to the JVM which will print an ugly stack trace. We even didn't bother to close the FileStream properly: when the JVM exits the FileReader will be closed anyway.

I leave it up to your imagination and creativity to implement a proper, industrial strength driver. The one above serves fine for demonstration purposes in this article.

I created a file "/sudoku.txt" with the following content I found in a newspaper:

Java Code: 
+-------+-------+-------+
| 7 . . | . . . | 4 . . | 
| . 2 . | . 7 . | . 8 . |
| . . 3 | . . 8 | . . 9 |
+-------+-------+-------+ 
| . . . | 5 . . | 3 . . |
| . 6 . | . 2 . | . 9 . |
| . . 1 | . . 7 | . . 6 |
+-------+-------+-------+ 
| . . . | 3 . . | 9 . . |
| . 3 . | . 4 . | . 6 . |
| . . 9 | . . 1 | . . 5 |
+-------+-------+-------+
Then I started the Sudoku class:

Java Code: 
java -classpath . Sudoku /sudoku.txt
And this is what was printed almost immediately:

Java Code: 
+-------+-------+-------+
| 7 9 8 | 6 3 5 | 4 2 1 | 
| 1 2 6 | 9 7 4 | 5 8 3 | 
| 4 5 3 | 2 1 8 | 6 7 9 | 
+-------+-------+-------+
| 9 7 2 | 5 8 6 | 3 1 4 | 
| 5 6 4 | 1 2 3 | 8 9 7 | 
| 3 8 1 | 4 9 7 | 2 5 6 | 
+-------+-------+-------+
| 6 1 7 | 3 5 2 | 9 4 8 | 
| 8 3 5 | 7 4 9 | 1 6 2 | 
| 2 4 9 | 8 6 1 | 7 3 5 | 
+-------+-------+-------+
I checked the next days' newpaper and indeed: both the newspaper as well as the solver showed this same solution. Just for fun I changed the content of my /sudoku.txt file:

Java Code: 
+-------+-------+-------+
| . . . | . . . | . . . | 
| . . . | . . . | . . . |
| . . . | . . . | . . . |
+-------+-------+-------+ 
| . . . | . . . | . . . |
| . . . | . . . | . . . |
| . . . | . . . | . . . |
+-------+-------+-------+ 
| . . . | . . . | . . . |
| . . . | . . . | . . . |
| . . . | . . . | . . . |
+-------+-------+-------+
and fired up the solver again; this was the output:

Java Code: 
+-------+-------+-------+
| 1 2 3 | 4 5 6 | 7 8 9 | 
| 4 5 6 | 7 8 9 | 1 2 3 | 
| 7 8 9 | 1 2 3 | 4 5 6 | 
+-------+-------+-------+
| 2 1 4 | 3 6 5 | 8 9 7 | 
| 3 6 5 | 8 9 7 | 2 1 4 | 
| 8 9 7 | 2 1 4 | 3 6 5 | 
+-------+-------+-------+
| 5 3 1 | 6 4 2 | 9 7 8 | 
| 6 4 2 | 9 7 8 | 5 3 1 | 
| 9 7 8 | 5 3 1 | 6 4 2 | 
+-------+-------+-------+
The output clearly shows that the solver works its way through the problem in a strictly left to right, top to bottom manner: check the first row and the first sub-square and see the regularity.

This is a naive solver but it works fine for the problems I tried. Some problems are considered to be extremely difficult. Play with this solver a bit, feed it some of those difficult problems and see how it runs.

The Sudoku class only needs a Reader and a Writer, i.e. it doesn't care how these streams were created. You can even create a free Sudoku solver server out of it if you feel like it: wrap a Reader around an InputStream created by a socket. Do the same for the Writer and the socket's OutputStream.

This solver finds the 'first' solution given a problem. You might try and alter the 'solve' method a bit so that it finds all solutions. It's up to you.

Today the Sudoku madness went a bit down, but I can still see those silly puzzles in the newspapers. I don't care anymore, I have my Sudoku solver ;-)

kind regards,

Jos ]]> JosAH http://www.java-forums.org/blogs/josah/75-sudoku.html Script Engines http://www.java-forums.org/blogs/josah/71-script-engines.html Sun, 12 Jun 2011 08:07:09 GMT Greetings,

Introduction

Java is not Javascript and most of the times when questions end up in the wrong forum, they're moved to another forum as soon as possible. Accidentally this article talks about Javascript a bit but it belongs in this Java section.

Since version 1.6 of Java a script engine comes packaged with the core classes. A script engine is an abstract little framework that offers support for other languages, scripting languages to be exact, a.k.a. interpreters.

Java version 1.6. comes bundled with one scripting language: Javascript. The next paragraphs decribe what you can do with this little framework and how you can do it.

ScriptEngineManager

A ScriptEngineManager manages (sic) ScriptEngines. A ScriptEngine is Java's interface to a scripting language. The manager doesn't know how to install, or instantiate, an engine itself: it uses so called ScriptEngineFactories for that purpose. So if you want to obtain a ScriptEngine you ask the manager for it; the manager checks if one of its factories can instantiate the appropriate engine; if so, it asks the factory to do so and the manager returns the new ScriptEngine to the caller.

Note that in this simple case we, the programmers, didn't need to deal with the factories ourself. Here's a simple example

Java Code: 
ScriptEngineManager manager= new ScriptEngineManager();
ScriptEngine engine= manager.getEngineByName("JavaScript");
try {
	engine.eval("print('Hello, world!')");
} catch (ScriptException ex) {
	ex.printStackTrace();
}
The first line instantiates the new manager. Line two indirectly instantiates a ScriptEngine for the Javascript language. The next lines put that ScriptEngine to work a bit; it's the obligatory "hello world!" program written in Javascript and executed (or "interpreted") from the Java environment.

ScriptEngineFactory

The factories are implementations of this interface. Factories can produce the ScriptEngines (see previous paragraph). Let's see what those factories can tell us; here's an example:

Java Code: 
ScriptEngineManager manager= new ScriptEngineManager();
List<ScriptEngineFactory> factories= manager.getEngineFactories();

for (ScriptEngineFactory factory: factories) {

	String name = factory.getEngineName();
	String version = factory.getEngineVersion();
	String language = factory.getLanguageName();

	System.out.println(name+"("+version+"): "+language);

	for(String n: factory.getNames()) 
      		System.out.println(n);
}
This example instantiates a ScriptEngineManager again and asks it for a List of available ScriptEngineFactories. For every factory its information is retrieved and printed. Note that a language doesn't just have one name, the language is also known by zero or more aliases; the inner loop shows the alias names too, if available.

Read the API documentation to see what more a ScriptEngineFactory can do for you. When I run that little snippet on my laptop this is the output:

Java Code: 
Mozilla Rhino(1.6 release 2): ECMAScript
js
rhino
JavaScript
javascript
ECMAScript
ecmascript
There's only one factory installed for the 'ECMAScript' language; that's the old name of Javascript; the ScriptEngine implementation is Mozilla Rhino. The language is also available under the alias names, "js", "rhino", "Javascript", and a few other variations.

How does the ScriptEngineManager know which ScriptEngineFactories are available? It uses quite a new naming convention for that: the jars that contain factories (and the engines) must be stored in a special directory and the manager checks that directory for the jars and dynamically figures out which factories are in those jars. A detailed discussion of this mechanism is beyond the scope of this article. Maybe in the near future I'll build a factory and an engine for the little expression language presented in a previous article series.

ScriptEngine

A ScriptEngine is Java's gateway to a particular script interpreter. But where does that script come from? There are two ways to feed a script to the engine: pass it a Reader or pass it a String. When reading from the Reader the script is supposed to be read. The Reader can be any Reader: wrapped around a socket InputStream, or maybe just a FileReader. The script can come from anywhere. The alternative is just to pass the entire script text as a String.

A script maintains 'bindings'. A binding is nothing more than a Map<String, Object>, i.e. it associated Strings with objects. The engine uses those bindings for its scripts. Here's a small example:

Java Code: 
ScriptEngineManager manager = new ScriptEngineManager();
ScriptEngine engine = manager.getEngineByName("JavaScript");
        
try {
	String expression = "a+b";
	engine.put("a", 41);
	engine.put("b", 1);
	Object result = engine.eval(expression);
	System.out.println(expression+"= "+result);
	System.out.println("type: "+result.getClass().getCanonicalName());
} catch(ScriptException se) {
	se.printStackTrace();
}
Both 'engine.put()' method calls put a new binding in the engine's bindings: a=41 and b=1. When I run this code snippet on my laptop I see this:

Java Code: 
a+b= 42.0
type: java.lang.Double
That ScriptEngine was smart enough to convert a Javascript '42' to a Java Double object. It was also smart enough to convert Java's ints '41' and '1' to the correct objects for Javascript so that it can evaluate 'a+b'. That's cute.

As a matter of fact, a ScriptEngine can convert all sorts of Java objects to the script's representation thereof and back again to Java's representation.

The Javascript engine can even directly use Java objects; I bluntly 'borrowed' the following example from a piece of Sun's text on the same topic:

Java Code: 
ScriptEngineManager manager = new ScriptEngineManager();
ScriptEngine engine = manager.getEngineByName("JavaScript");
        
try {
	engine.eval(
		"importPackage(javax.swing);" +
		"var pane = " +
		"JOptionPane.showMessageDialog(null, 'Hello, world!');");
} catch (ScriptException ex) {
	ex.printStackTrace();
}
Invocable

Script engines don't just read, parse and interpret source text; they compile
the script text to their internal form. Such engines implement another interface,
the Invocable interface.

Here's an example showing how this interface can be used:

Java Code: 
ScriptEngineManager manager = new ScriptEngineManager();
ScriptEngine engine = manager.getEngineByName("JavaScript");
        
try {
	String expression = "function add(x, y) { return x+y; }";
	engine.eval(expression);
           
	engine.put("a", 41);
	engine.put("b", 1);

	Invocable invocable= (Invocable)engine;
            
	System.out.println("add(1, 2)= "+invocable.invokeFunction("add", 1, 2));
	System.out.println("add(a, b)= "+invocable.invokeFunction("add", "a", "b"));
	System.out.println("add(a, b)= "+engine.eval("add(a, b)"));

} catch(ScriptException se) {
	se.printStackTrace();
} catch (NoSuchMethodException nsme) {
	nsme.printStackTrace();
}
First the small script "function add(x, y) { return x+y; }' is compiled and two bindings are added. Next the ScriptEngine is cast to an Invocable. If the cast fails (it doesn't in this example), an exception is thrown indicating that the particular script cannot compile the script and keep it for later use.

When I ran this little code snippet, this was the output:

Java Code: 
add(1, 2)= 3.0
add(a, b)= ab
add(a, b)= 42.0
Note that the second invocation tried to add the String values "a" and "b", not the values present in the bindings. The result is the String "ab" showing that Javascript concatenates strings when the '+' operator is applied to them.

The third call properly uses the bound values for a and b again.

Concluding remarks

There's much more that can be done with this quite new little framework. The framework is the result of all the work done by JSR 223 and at this very moment they're working on implementations for the Ruby language, the BeanShell interpreter and other languages as well. This article showed the basic usage and the structure of the ScriptEngine framework. Have fun with it.

kind regards,

Jos ]]> JosAH http://www.java-forums.org/blogs/josah/71-script-engines.html Heap Sort II http://www.java-forums.org/blogs/josah/70-heap-sort-ii.html Sun, 12 Jun 2011 07:54:54 GMT Greetings,

This is the second part of the Heap Sort article (that dumb forum software only allows 10000 characters per article part. Here's the second part:

The first part of this article explained a bit about the heap sort algorithm itself. Now let's get back to our original problem: given the two arrays:

Java Code: 
String[] first= { "Fred", "Barney", "Wilma", "Betty", "Bambam", "Pebbles" };
String[] last= { "Flintstone", "Rubble", "FlintStone", "Rubble", "Rubble", "Flintstone" };
We have to create a Sortable that can return the number of first/last names, it can compare two first names (for example) and it can swap two elements in both the first and last name arrays. We need to sort these two arrays according to the first name; the other array just needs to 'follow' i.e its last names must correspond to the first names after the sort has finished.

Here's the complete example without many comments:

Java Code: 
public class Flintstones implements Sortable {

	private String[] first;
	private String[] last;

	public FlintStones(String[] first, String[] last) {
	
		this.first= first;
		this.last = last;
	}

	public int length() { // interface implementation

		return first.length; 
	} 

	public int compare(int i, int j) { // interface implementation

		return first[i].compareTo(first[j]);
	}

	private void swap (String[] s, int i, int j) { // private helper method
	
		String t= s[i]; s[i]= s[j]; s[j]= t;
		
	}

	public void swap(int i, int j) { // interface implementation
		
		swap(first, i, j);
		swap(last, i, j);
	}

	public static void main(String[] args) {

		String[] first= { "Fred", "Barney", "Wilma", "Betty", "Bambam", "Pebbles" };
		String[] last= { "Flintstone", "Rubble", "FlintStone", "Rubble", "Rubble", "Flintstone" };
		
		HeapSort.sort(new FlintStones(first, last));

		for (int i= 0; i < first.length; i++)
			System.out.println(first[i]+" "+last[i]);
	}
}
So what was this artice all about? It showed that you can sort not just arrays or Lists, but anything that implements the Sortable interface. It also showed some of the gory details of the heap sort algorithm. This Sortable interface can even be implemented by something that stores its data on disk.

kind regards,

Jos ]]> JosAH http://www.java-forums.org/blogs/josah/70-heap-sort-ii.html Heap Sort I http://www.java-forums.org/blogs/josah/69-heap-sort-i.html Sun, 12 Jun 2011 07:52:03 GMT Greetings, A lot of topics are started here in this forum (and a lot of other forums too) mentioning a problem about sorting data. Examples of... Greetings,

A lot of topics are started here in this forum (and a lot of other forums too) mentioning a problem about sorting data.

Examples of two main problem scenarios are like this:

1) an array containing a first name and another array containing a last name and possibly a third array containing the age of a person; how to sort the arrays according to, say, the first name.

2) a List of objects where each object contains a first name, a last name and an age. The list must be sorted according to, say, the last name. The second scenario can easily be solved if you implement a Comparator; something like the following will do fine:

Java Code: 
public class PersonComparator implements Comparator<Person> {
	public int compare(Person l, Person r) {
		return l.getLastName().compareTo(r.getLastName());
	}
}
Note that according to the contract of the Comparable<T> interface just a single method needs to be implemented. The equals() method can be left out, possibly at the cost of a small performance penalty.

You need to pass an instance of such a PersonComparator to either a:

a) Collections.sort()
b) Arrays.sort()

method, i.e. the first method sorts a List using a Comparator for the nitty gritty comparison work, or, the second method sorts an array using this Comparator.

But what about scenario 1 then? Of course scenario 1 was bad design to start with but also of course it wasn't your design, it was a bit of legacy code and you weren't around when the design's first implementation was deployed.
Even worse, the solution to scenario 2 only works for arrays or Lists. Scenario 2 is like scenario 1 for other datastructures that need to be sorted according to some sorting criteria. Those two methods (although useful) aren't of much help for anything else that needs to be sorted.

Again there are two possible scenarios here (I call them A and B): either

A) transmogrify your data to an array or List and use the methods mentioned above; after that, re-transmogrify the sorted data to your original data and go on.

or

B) come up with something else.

This little article is all about scenario B, i.e. we come up with something else. What exactly do we need to know about any type of data to be sortable? First we need to know how many data items are to be sorted. We also need to know whether or not one data item is larger, equal or smaller than another data item. Third we want to be able to interchange (or swap) two data items. Given the first notion we'd like to be able to think of indices, i.e. data item 0, data item 1 etc. etc.

The above notions simply beg for an interface definition. Here is a simple one:

Java Code: 
public interface Sortable {
	int length();
	int compare(int i, int j);
	void swap(int i, int j);
}
Note that, no matter how trivial this interface may look, if a sorting method is able to sort a Sortable, it doesn't need to know anything at all about the type of data it sorts and it doesn't care less how two data items are swapped and it doesn't care less why one data item is considered to be less, equal or greater than an other data item.

This little article shows a 'heap sort' sorting method that is very well capable of doing the job. A heapsort method is no worse than the ever propagated quick sort method and it runs around any other naive sorting methods big times.

The big-Oh of a heap sort method is equal or better than a quick sort method and as an additional benefit it is stable w.r.t. that (in)famous big-Oh number. Before I forget, it doesn't need any additional memory proportional to the size of the data to be sorted.

This article presents the heap sort algoritm in terms of the Sortable interface. So when two data items need to be compared, the algorithm calls the compare() method, when they need to be swapped, the Sortable can do it etc. etc.

Here is a definition:

a heap is a binary tree such that for any node P, its children L and R (if present) are not less than P, i.e. compare(P, L) and compare(P, R) both return a value >= 0.

Note that a leaf is a heap by itself (a leaf doesn't have children)

For (almost) no particular reason, here's a binary tree with ten nodes:
Java Code: 
                  0
               /     \
              1       2        
            /   \   /   \
           3    4   5    6
          / \  / 
          7 8  9
please forgive me my (lack of) ASCII art. This tree does show some interesting properties (without proof):

1: the left kid of a node i is numbered 2*i+1;
2: the right kid of a node i is numbered 2*i+2;
3: if a node i > 0 has k kids, a node i-1 has at least k kids;
4: every node has as many kids as possible.

Those four notions define a 'complete' binary tree. Math folks like those definitions. So a sequence 0, 1, 2 ... n-1 can be seen as a complete binary tree.

Back to that heapsort thing again: suppose that a node P is larger than both of its children L and R for every node in a tree (as sketched above). Then the root of the tree is the largest element of the them all.

All leaves of a tree are heaps by definition, but what about the rest of the tree nodes (nearer to the root of the tree)? The solution is simple: we can build a heap out of a binary tree, given the fact that both the left and right subtrees are heaps already; it works as follows:

Let P be a parent node and L and R the left and right children of P.

1: if P >= L and P >= R then we already have a heap, otherwise:
2: let M be the maximum of L and R;
3: swap P with that maximum M and continue from step 1 again with P at its new position.

For every complete binary tree with n nodes we know that the last n/2 nodes of that tree are leaves and so they are heaps already. (Check this by using my bad little ASCII art above).

So we can turn a (sub)tree into a heap using the three line algoritm above. Lets do it using the Sortable interface defined earlier; here goes:

Java Code: 
// given a Sortable, build a heap starting at node p. There a n nodes in total
private static void heapify(Sortable s, int p, int n) {
		
	for (int r, l= (p<<1)+1; l < n; p= l, l= (p<<1)+1) {
		
		// l is the maximum of l and r, the two subnodes of p 
		if ((r= l+1) < n && s.compare(l, r) < 0) l= r;
			
		// check if parent p is less than maximum l
		if (s.compare(p, l) < 0) s.swap(p, l);
		else break;
	}
}
This is all fine, but how to turn a Sortable into a heap? We'll just use the small method above. Traditionally this part is called the 'phase 1' of the heap sort algorithm. Why break with good traditions?

Java Code: 
private static void phase1(Sortable s) {

	// heapify all the non-leaf nodes 		
	for (int n= s.length(), p= n/2; p >= 0; p--)
		heapify(s, p, n);
}
The second phase, traditionally named 'phase 2' *ahem* uses a heap as follows to get everything sorted. The root of the tree is the largest element of all the nodes. If we swap the root with the last element of the Sortable we have this largest element in place (at the end). But now the tree is probably not a heap anymore. Using our first method we can turn the tree into a heap again but we ignore the last node (which was in place already).

We repeat the step above until the heap contains just one single node. This node must be the smallest element of all elements in the Sortable. Here goes:

Java Code: 
// sort the Sortable
private static void phase2(Sortable s) {
		
	for (int n= s.length(); --n > 0; ) {
		s.swap(0, n); 		// put the root element in its place
		heapify(s, 0, n); 	// and restore the heap again
	}		
}
I don't think it comes as a surprise that the heap sort method itself simply looks like this:

Java Code: 
// driver for the methods
public static Sortable sort(Sortable s) { 
		
	phase1(s); 	// build initial heap
	phase2(s); 	// sort the sortable given the heap
		
	return s; 	// return the Sortable for convenience
}
For completeness, so you can copy and paste this sorting utility straight in your own private back of tricks, here is the entire class:

Java Code: 
public final class HeapSort {

	private HeapSort() { } // no istances of this class possible

	// given a Sortable, build a heap starting at node i. There a n nodes in total
	private static void heapify(Sortable s, int p, int n) {
		
		for (int r, l= (p<<1)+1; l < n; p= l, l= (p<<1)+1) {
		
			// l is the maximum of l and r, the two subnodes of p 
			if ((r= l+1) < n && s.compare(l, r) < 0) l= r;
		
			// check if parent p is less than maximum l
			if (s.compare(p, l) < 0) s.swap(p, l);
			else break;
		}

	// build a heap out of the Sortable in place
	private static void phase1(Sortable s) {

		// heapify all the non-leaf nodes 		
		for (int n= s.length(), p= n/2; p >= 0; p--)
			heapify(s, p, n);
	}

	// sort the Sortable
	private static void phase2(Sortable s) {
		
		for (int n= s.length(); --n > 0; ) {
			s.swap(0, n); 		// put the root element in its place
			heapify(s, 0, n); 	// and restore the heap again
		}		
	}

	// driver for the worked methods
	public static Sortable sort(Sortable s) { 
		
		phase1(s); 	// build initial heap
		phase2(s); 	// sort the sortable given the heap
		
		return s; 	// return the Sortable for convenience
	}
}
This class takes about one page of text in my VI text editor. Let's step back a bit from this code and realize what we've just created: this sort algoritm is very efficient and stable w.r.t. it's runtime big-Oh wise. This implementation doesn't know anything about what exactly needs to be sorted, i.e. it only uses the Sortable interface.

The class is final and contains a private default constructor so there is no way for anyone else to create an instance of a HeapSort object; because no such object is needed: this class is a utility class with just functionality.

kind regards,

Jos ]]> JosAH http://www.java-forums.org/blogs/josah/69-heap-sort-i.html