Java Programming Forum - Learn Java Programming - Blogs - JMeter

Using Variables to parameterise tests

JMeter — Fri, 27 Apr 2012 17:29:44 GMT

Variables don't have to vary - they can be defined once, and if left alone, will not change value. So you can use them as short-hand for expressions that appear frequently in a test plan. Or for items which are constant during a run, but which may vary between runs. For example, the name of a host, or the number of threads in a thread group.

When deciding how to structure a Test Plan, make a note of which items are constant for the run, but which may change between runs. Decide on some variable names for these - perhaps use a naming convention such as prefixing them with C_ or K_ or using uppercase only to distinguish them from variables that need to change during the test. Also consider which items need to be local to a thread - for example counters or values extracted with the Regular Expression Post-Processor. You may wish to use a different naming convention for these.

Properties and Variables

JMeter — Fri, 27 Apr 2012 17:29:01 GMT

JMeter properties are defined in jmeter.properties. Properties are global to jmeter, and are mostly used to define some of the defaults JMeter uses. For example the property remote_hosts defines the servers that JMeter will try to run remotely. Properties can be referenced in test plans.

JMeter variables are local to each thread. The values may be the same for each thread, or they may be different.
If a variable is updated by a thread, only the thread copy of the variable is changed. For example the Regular Expression Extractor Post-Processor will set its variables according to the sample that its thread has read, and these can be used later by the same thread.

Note that the values defined by the Test Plan and the User Defined Variables configuration element are made available to the whole test plan at startup. If the same variable is defined by multiple UDV elements, then the last one takes effect. Once a thread has started, the initial set of variables is copied to each thread. Other elements such as the User Parameters Pre-Processor or Regular Expression Extractor Post-Processor may be used to redefine the same variables (or create new ones). These redefinitions only apply to the current thread.

JMeter Assertions

JMeter — Fri, 27 Apr 2012 17:28:12 GMT

Assertions allow you to assert facts about responses received from the server being tested. Using an assertion, you can essentially "test" that your application is returning the results you expect it to.
For instance, you can assert that the response to a query will contain some particular text. The text you specify can be a Perl-style regular expression, and you can indicate that the response is to contain the text, or that it should match the whole response.

You can add an assertion to any Sampler. For example, you can add an assertion to a HTTP Request that checks for the text, "". JMeter will then check that the text is present in the HTTP response. If JMeter cannot find the text, then it will mark this as a failed request.

To view the assertion results, add an Assertion Listener to the Thread Group. Failed Assertions will also show up in the Tree View and Table Listeners, and will count towards the error %age for example in the Aggregate and Summary reports.

JMeter Timers

JMeter — Fri, 27 Apr 2012 17:27:30 GMT

By default, a JMeter thread sends requests without pausing between each request. We recommend that you specify a delay by adding one of the available timers to your Thread Group. If you do not add a delay, JMeter could overwhelm your server by making too many requests in a very short amount of time.

The timer will cause JMeter to delay a certain amount of time before each sampler which is in its scope.

If you choose to add more than one timer to a Thread Group, JMeter takes the sum of the timers and pauses for that amount of time before executing the samplers to which the timers apply. Timers can be added as children of samplers or controllers in order to restrict the samplers to which they are applied.

JMeter Listeners

JMeter — Fri, 27 Apr 2012 17:26:44 GMT

Listeners provide access to the information JMeter gathers about the test cases while JMeter runs. The Graph Results listener plots the response times on a graph. The "View Results Tree" Listener shows details of sampler requests and responses, and can display basic HTML and XML representations of the response. Other listeners provide summary or aggregation information.

Additionally, listeners can direct the data to a file for later use. Every listener in JMeter provides a field to indicate the file to store data to. There is also a Configuration button which can be used to choose which fields to save, and whether to use CSV or XML format. Note that all Listeners save the same data; the only difference is in the way the data is presented on the screen.

Listeners can be added anywhere in the test, including directly under the test plan. They will collect data only from elements at or below their level.

Error reporting

JMeter — Fri, 27 Apr 2012 17:26:08 GMT

JMeter reports warnings and errors to the jmeter.log file, as well as some information on the test run itself. Just occasionally there may be some errors that JMeter is unable to trap and log; these will appear on the command console. If a test is not behaving as you expect, please check the log file in case any errors have been reported (e.g. perhaps a syntax error in a function call).

Sampling errors (e.g. HTTP 404 - file not found) are not normally reported in the log file. Instead these are stored as attributes of the sample result. The status of a sample result can be seen in the various different Listeners.

Stopping a Test

JMeter — Fri, 27 Apr 2012 17:25:31 GMT

There are two types of stop command available from the menu:

� Stop (Control + '.') - stops the threads immediately if possible. In Versions of JMeter after 2.3.2, many samplers are now Interruptible which means that active samples can be terminated early. The stop command will check that all threads have stopped within the default timeout, which is 5000 ms = 5 seconds. [This can be changed using the JMeter property jmeterengine.threadstop.wait ] If the threads have not stopped, then a message is displayed. The Stop command can be retried, but if it fails, then it is necessary to exit JMeter to clean up.
� Shutdown (Control + ',')- requests the threads to stop at the end of any current work. Will not interrupt any active samples. The modal shutdown dialog box will remain active until all threads have stopped.
Versions of JMeter after 2.3.2 allow a Stop to be initiated if Shutdown is taking too long. Close the Shutdown dialog box and select Run/Stop, or just press Control + '.'.

Running a Test Plan

JMeter — Fri, 27 Apr 2012 17:24:47 GMT

To run your test plan, choose "Start" (Control + r) from the "Run" menu item. When JMeter is running, it shows a small green box at the right hand end of the section just under the menu bar. You can also check the "Run" menu. If "Start" is disabled, and "Stop" is enabled, then JMeter is running your test plan (or, at least, it thinks it is).

The numbers to the left of the green box are the number of active threads / total number of threads. These only apply to a locally run test; they do not include any threads started on remote systems when using client-server mode.

Running JMeter

JMeter — Fri, 27 Apr 2012 17:24:11 GMT

To run JMeter, run the jmeter.bat (for Windows) or jmeter (for Unix) file. These files are found in the bin directory. After a short pause, the JMeter GUI should appear.

There are some additional scripts in the bin directory that you may find useful. Windows script files (the .CMD files require Win2K or later):

� jmeter.bat - run JMeter (in GUI mode by default)
� jmeter-n.cmd - drop a JMX file on this to run a non-GUI test
� jmeter-n-r.cmd - drop a JMX file on this to run a non-GUI test remotely
� jmeter-t.cmd - drop a JMX file on this to load it in GUI mode
� jmeter-server.bat - start JMeter in server mode
� mirror-server.cmd - runs the JMeter Mirror Server in non-GUI mode
� shutdown.cmd - Run the Shutdown client to stop a non-GUI instance gracefully
� stoptest.cmd - Run the Shutdown client to stop a non-GUI instance abruptly

JMeter's Classpath

JMeter — Fri, 27 Apr 2012 17:23:34 GMT

The classes from jars are automatically found by the Jmeter in the following directories:

JMETER_HOME/lib - used for utility jars
JMETER_HOME/lib/ext - used for JMeter components

One point to remember here is that you should jar any JMeter components developed by you and also you need to copy the jar into JMeter's lib/ext directory. JMeter components will be automatically found by the JMeter in any jars found here.

Furthermore you should define the property search_ paths in jmeter.properties in case you don't want to put the extension jars in the lib/ext directory. And also lib/ext is intended for JMeter components only and not for utility jars so don't use lib/ext for utility jars. Remember that other jars should be placed in the lib directory - not the lib/ext directory, jars like, JavaMail, JDBC, Beanshell etc.

The principle of JMeter

JMeter — Fri, 27 Apr 2012 17:22:52 GMT

The principle of JMeter is very simple. If you want to test e.g. a SOAP interface layer, all you basically need is the URL and SOAP request. Starting with that you can build your test plan. And this can be as fancy as you want. Using variables, counters, parameters, CSV files, loops, logs, etc. There are almost no limits in designing your test and making it as maintainable as possible.

The principle of JMeter

JMeter Installation

JMeter — Fri, 27 Apr 2012 17:21:20 GMT

The installation of JMeter (if you would call it an installation) is pretty straight forward. On the website is a link to the download area of stable versions. You also have the possibility to use nightly builds, but this is at your own risk. No guarantee that they work properly. So the advice is always to start with a stable version.

Download the latest version (zip or tgz) and unpack the archive to a local folder. Before starting JMeter it is wise to have a look at the configuration. The jmeter.properties file (located in the bin folder) contains a lot of settings. Most of those settings should be fine for the average user. The file is well documented and easy to read (and change where needed).

Also pay attention to needed additional jars. If you are going to use for example JDBC, JMS or JavaMail additional jars are not included in the JMeter installation. The user guide on the website explains how to use it and where to get it.

Starting JMeter is just a matter of double clicking jmeter.bat in the bin folder.

Apache JMeter Features

JMeter — Fri, 27 Apr 2012 17:20:27 GMT

Apache JMeter features include:
� Can load and performance test many different server types:

o Web - HTTP, HTTPS
o SOAP
o Database via JDBC
o LDAP
o JMS
o Mail - POP3(S) and IMAP(S)
� Complete portability and 100% Java purity .
� Full multithreading framework allows concurrent sampling by many threads and simultaneous sampling of different functions by separate thread groups.
� Careful GUI design allows faster operation and more precise timings.
� Caching and offline analysis/replaying of test results.
� Highly Extensible:

o Pluggable Samplers allow unlimited testing capabilities.
o Several load statistics may be choosen with pluggable timers .
o Data analysis and visualization plugins allow great extensibility as well as personalization.
o Functions can be used to provide dynamic input to a test or provide data manipulation.
o Scriptable Samplers (BeanShell is fully supported; and there is a sampler which supports BSF-compatible languages)

Apache JMeter

JMeter — Fri, 27 Apr 2012 17:19:39 GMT

The Apache JMeter� desktop application is open source software, a 100% pure Java application designed to load test functional behavior and measure performance. It was originally designed for testing Web Applications but has since expanded to other test functions.

Apache JMeter may be used to test performance both on static and dynamic resources (files, Servlets, Perl scripts, Java Objects, Data Bases and Queries, FTP Servers and more). It can be used to simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types. You can use it to make a graphical analysis of performance or to test your server/script/object behavior under heavy concurrent load.