Tips and Techniques for Working with the Rules Editor

The rules editor is designed to be easy to use. There are however a few tricks that will make using the rules editor even easier. The following covers a few of these tricks.

Selecting multiple rules

If you need to move multiple rules around in unison or select them so that you can copy them to another rule set, hold down the CTRL key whilst clicking and selecting rules. Alternatively, you can hold down the mouse key and drag a rectangle around the rules you wish to select.

Cut, copy and paste

Rules can be cut, copy or pasted within the same rule set or between open rule sets by right-clicking whilst the rules are selected. To paste the rules into a new position, right-click on the canvas where the rules should be placed and select Paste.

Note: Not all browsers support the right-click feature. For this reason, edit options can also be obtained by holding down shift and left-click.

Disconnecting chains

If you mistakenly connect one rule to another, you can remove the connection by right-clicking on the chain point and selecting Disconnect.

Distinguishing between variables, text and numbers

Many of the rules in the system take a variable, text or a number as a parameter. Generally speaking, variables may only contain letters, numbers and the characters: ‘_’, ‘:’ and ‘.’, and they must start with a letter.

There will be times where a variable can be confused with a text literal, and for that reason text literals should always be enclosed in double quotes. For example: ABC is a variable whereas “ABC” is the text ABC.

Numbers on the other hand are unambiguous and can just be keyed as numbers.

When entering a CSV list of values, there is no need to enclose the entire block in double quotes. Since CSV text has commas in it, it will automatically be detected as a list of string elements.

Don't close before testing

There is no need to close a rule set to test it. You can keep multiple rule sets open in multiple windows, deploy your rules, test and then return to the already open windows to continue editing. Click the save button rather than the close button to stay on the page.

Regular expressions

Several of the standard rules make use of a string matching feature known as regular expressions. Books have been written about regular expressions and it is beyond the scope of this manual to cover more than the very basics.

In its most basic form, you can use regular expressions to see if a certain text is available within another, to count characters and to look for certain pieces of text at certain positions within words.

An example of a regular expression would be: (ab|cd)

This expression will check if a text contains the character sequence “ab” or the sequence “cd”. So both “baby” and “lcd” would be a match.

The following tables list some of the common uses of regular expressions and how they can be used to validate text.

Character ranges

Character(s)	Description	Example	Matches	Does not match
[]	Matches any of the enclosed characters	AB[cC]	ABC, ABc	abC, def
[^]	Does not match any of the enclosed characters	AB[^cC]	ABD, ABz	ABc, ABC
[a-z]	Range	AB[C-F]	ABC, ABD, ABF	ABG, ABc
.	Any character except newline	N.	N1, N2	N
\d	Any digit between 0 and 9	N\d	N1, N2	NA
\D	Any non-digit character	N\d	NA	N1, N2
\w	Any letter, digit or underscore (equal to [a-zA-Z0-9_])	ab\wd	abcd, abCd, abad	Ab/1d
\W	Any character other than a letter, digit or underscore (equal to [^a-zA-Z0-9_])	ab\Wd	Ab/1d	abcd, abCd, abad
\s	Any single white space character (space, new line etc.)	ab\sde	ab de	abde
\S	Any single non-white space character	ab\Sde	abcde	ab de

Counting

Character(s)	Description	Example	Matches	Does not match
*	Zero or more occurrences	Tes*	Test, Television, Tess	Two, Trees
?	Zero or one occurrence	Colou?r	Color, Colour	Colouur
+	One or more occurrences	Tes+	Test, Tess	Television
{x}	Exactly x occurrences of the previous character	Hel{2}o	Hello	Helo,Helllo
{x,}	x or more occurrences of the previous character	Hel{2,}o{2,}	Helllloooo	Hello, Heloo, Howdy
{x,y}	Between x and y occurrences of the previous character	Hel{2,3}o	Hello, Helllo	Helo, Hellllo

Grouping and alternation

Character(s)	Description	Example	Matches	Does not match
()	Group together	(abc){2,}	abcabc	abc
x|y	Either x or y	(abc|def){2,}	defabc, abcabc, ghjabc	abc

Positioning

Character(s)	Description	Example	Matches	Does not match
^	Match at beginning of string	^hello	hello world	say hello
$	Match at end of string	hello$	say hello	hello world
\b	Match at beginning or end of a word	\bhel	say hello, hello world	Chellos
\B	Match inside a word	\Bhel	Chellos	say hello, hello world
(?=x)	Only if followed by x	ab(?=c)	abc	ab, abd
(?!x)	Only if not followed by x	ab(?!c)	ab, abd	abc

Lists and Data Sets

Lists and data sets provide an efficient way to work with keyed data that needs to be stored either in memory or a database.

Lists are capable of storing normal variables, lists or data sets, allowing effectively for multi-dimensional arrays.

There are two types of lists: Regular lists and fixed sized lists. When you insert a value into a list, the list will automatically be created as a regular list if it hasn’t already been created.

Fixed size lists are extremely useful for memory caching. Elements inserted into a fixed size list will stay there until the maximum size of the list is reached. Once that happens, the oldest element that has not been accessed (read or updated) will be removed from the list.

Elements inserted into a list must have a fixed key. If a new element is inserted with an existing key, the existing element will be replaced.

It is possible to create global lists by creating the list in a startup rule set and then setting it as a global variable. When the global list is read from a normal rule set, any changes made to that list as a local variable will directly affect the global list. This provides a means for caching data at a global level.

Data sets provide a way to create a collection of correlated data. For example, you can define a data set called “Fruit” with the properties Name, Color and Shape. Once you have defined a data set, you can create instances of it in a database or in a list. For example: Apple, Red and Round.

The X Engine will automatically handle the correct storage of the data set in a database and properties of the data set can be added and/or removed at any time in its lifecycle. So, if at a later stage you need to store another property in your Fruit data set, you can simply add it to the definition.

Data sets should be defined in a start-up rule set and can only be defined once within the life cycle of a deployment.

Data sets can optionally contain a number of lists. However, lists stored within a data set may only be single dimensional (you cannot have lists within lists).

Once a data set is stored within a database or a list, you can read it by key, delete it, update it etc. To update a data set within a list, you simply create it again with the same key name.

Using the New Rules Wizard

A quick way to get started building rules for a new web application or for stress testing is to use the rules wizard. The rules wizard uses live test data to extract URIs visited and build a structured collection of rule sets.

Selecting or creating a repository

The rules wizard creates a large number of files, so we strongly recommend that you create a new repository to write the new files to.

Preparing the data for the wizard

Before using the rules wizard, you must first deploy and start either the RuleWizardConf or the StressTestConf configuration found in the Rules Wizard repository to a test server that is protecting the target application (alternatively you can use the zero installation test method which is covered later in the manual). Once done, simply begin navigating the various components of the target web application.

Once you have visited all of the pages you wish to cover with your initial rule set, return to the console and go to the server status screen:

Qwerty Demo Server

In this example the Qwerty demo application was chosen and there are 19 test records ready to be processed.

Creating the rule sets and configuration

Now click on the "New rules wizard" button. You will be presented with the following page that controls the wizard:

Create new rules wizard

At this point you can have the rules wizard create a filter for the rules that automatically exclude static content. It is a comma separated list and you can easily add new elements.

Make sure that you select either New rule set or Stress test, depending on your requirements.

When you have selected an appropriate repository (in our case, we have use the repository ‘Name’) and reviewed the exclusion list, click on "Create".

After a brief pause, the X Engine will write out a complete configuration and collection of rules. The following pages show how these rule sets are structured for a new rule set:

Understanding the Load page

In keeping with best practice for rules writing, the rules wizard always creates a "Load" rule set. This contains rules that are generic for all URIs. The name of the page will be the name of the repository followed by the word "Load".

NameLoad rule set

This rule set will first check for malformed HTTP requests and, if found, will reject them. Subsequently it filters out static content, adds a tracker rule and then proceeds to the main rule set.

Understanding the Main page

The main rule set page contains a basic structure that determines the URI being visited and then uses a switch to re-direct to a rule set covering that URI.

NameMain rule set

Each one of those rule sets, in turn, are blank, but are already created and ready to have rules added to them. The new rules wizard creates a quick foundation to get you started with writing rules for your application.

Using the page hints

When the rules wizard creates each of the blank templates it includes sample information of the fields gathered and their values in the page description:

Rule infos tab

You can use this information to identify the fields available to your rules.

Using the New Rules Wizard for stress testing

As described above, an alternative use of the New Rules Wizard is to create a set of rules for stress testing an application. To do this, you must first deploy the StressTestConf configuration from the Rule Wizard repository to the web application on the browser proxy.

Preparing the data for the stress test

Once deployed, you should work through the application to be stress tested, step by step. Try to complete pages as normally as possible, making sure not to pause unnecessarily during the process, as wait times are recorded.

During the new rules wizard creation, select Stress test instead of New rule set.

The following pages show how the rule sets are structured for a stress test:

Understanding the Load page

In keeping with best practice for rules writing, the rules wizard always creates a "Load" rule set. This contains rules that are generic for all URIs. The name of the page will be the name of the repository followed by the word "Load".

NameLoad rule set

This rule set allows you to set up a multitude of things. First of all, the target server identity, and also the user agent to use. It is very common to modify the Load rule set to pick up randomized values for elements such as users. The following shows a modified Load rule set to include diversified User ID and Password configurations for a Qwerty stress test:

Modified rule set for Load rule set

In this scenario, the users and their passwords are picked from a CSV file of valid entries. The settings for the CSV Line Picker rule are as follows:

props

It is important to notice how the THREAD_NO variable is used to pick a line in the CSV file. The THREAD_NO is incremented for every stress testing agent and should the number exceed the number of entries in the CSV file, the rule will start reading from the beginning again.

Understanding the Main page

The main rule set page contains a basic structure that determines the URI being visited and then uses a Number Sequencer rule to re-direct to a rule set covering that URI.

NameMain rule set

Understanding the Wait page

The wait rule set page is used to define wait times. This delay can be fixed, or it can be a function of the recorded wait time during rule set creation.

The following shows a wait time delay:

NameWait rule set

The wait time properties can be set as follows:

NameWait props

The default for the percentage random is zero, but it can be randomized to create a more realistic user load.

An alternative is to either remove the wait time completely or insert a fixed delay:

NameWait rule set

Understanding the Fail page

The fail page is used to define any action to be taken if a stress test page invocation fails. The default is to abort the flow and stop the thread:

NameFail rule set

Understanding each stress page

Each page that is detected will be assigned a unique page name and will be given a unique rule set. The following shows the main page in the Qwerty application:

Name_main rule set

In this case, the page is a GET and the invocation is relatively simple. For POST requests, the structure looks slightly different:

In this case, the recorded POST variables are set up in a single rule. You can override these POST variables to use pseudo random values (as shown previously in the Load rule set).

The individual pages are where you should include test data, and possibly review the response data from significant pages to ensure that the stress test is progressing well.