LogoLogo
  • Documentation
  • PRODUCT REFERENCE
    • Introduction
    • Concepts and Terminology
      • Console
      • Servers (X Agents)
      • Projects
      • Configurations
      • Rules Editor
      • Rules, rule catalogue, protocol rules and rule sets
      • Test Data
      • Trace Data
      • Flight Recorders
      • Case Managers
      • Data Files
      • Content Files
      • Performance Data
      • Extensions
      • Protocols
      • Credential Vault
      • Custom Functions
      • Databases
      • Input Adaptors
      • Users
      • User Roles
      • Access Rules
      • Repositories
      • Audit Log
      • Proxies
    • Architectural Scenarios
      • Command and Control
      • Simplest Form
      • Servlet Filter
      • API Transformation
      • Active Web Proxy
      • Web Application Server
      • Active Proxy With Content
      • Mobile Application Server
      • Asynchronous Multi-Protocol
      • Data Loss Prevention Architecture
    • Getting Started
      • Login
      • Essential Things to do First
      • Keeping the Product Current
      • Common Console Management Tasks
      • Viewing Active Servers
    • Quick Product Introduction
      • Preparing the Browser Proxy
      • Setting up the Proxy in the Browser
      • Verifying the Browser Configuration
      • Understanding the Configuration
      • Understanding input and variables
      • Preparing a new repository
      • Locating the Page to Modify
      • Determining the Actions Required
      • Building the First Rule Set
      • Setting Rule Properties
      • Connecting up the First Rule
      • Getting a Server Result
      • Manipulating the Server Result
      • Returning the Result to the User
      • Creating a Configuration for the Rule Set
      • Selecting the Input Source
      • Deploying the New Configuration
      • Testing the Rules
    • X Agent details
      • Configuration Settings
      • Rule Sets
      • Tips and Techniques for Working with the Rules Editor
      • Web Application Rule Set Patterns
      • Building Rule Sets for Inclusion
      • Notes on Working with SOAP
      • Zero Installation Rules Testing
    • Data Files
    • Content Files
      • Previewing Content Files
      • Editing Content Files
      • Deploying Content Files
    • Test Data
      • Uploading and Downloading Test Data
    • Performance Data
      • Setting the Performance Collection Level
      • Retrieving the Performance Data
      • Viewing Transaction Counts
      • Viewing Transaction Counts and Inline Time
      • Viewing the Complete Performance Report
      • Understanding the URI Performance Information
      • Performance Benchmarks
    • Trace Data
      • Understanding the Trace
    • Tracing the Result of a System Failure
    • Conjoined Performance and Trace data
    • Live Performance Data and Probes
      • Setting Probes
      • Live Performance Impact Considerations
    • Accessing Server Logs
      • Viewing a log
    • Working with Flight Recorders
      • Searching Flight Recorder Information
      • Retrieving Flight Recorder Data
      • Graphing Flight Recorder Data
    • Working with Case Managers
      • Case Manager Definitions
      • Navigating the Case Managers
      • Creating a Case From Rules
      • Creating a Case From the Console
      • Viewing the Queues
      • Picking a Task
      • Case View
      • Completing a Task
      • Task History
      • Searching Tasks
      • Searching Cases
    • Managing Repositories
      • Moving Files Between Repositories
      • Copying Configurations and Rule Sets Together
      • Copying Dependent Rule Sets
      • Backing up
      • Moving Entire Repositories Between Consoles
      • Setting Repository Imports
      • Specifying Repository Rule Group Restrictions
    • Version Control and Restoring Files
    • Using the Portal UI
      • Using the Start Menu
      • Positioning Windows
      • Resizing Windows
      • Maximize, Restore and Minimize
      • Storing and Switching Desktops
    • Enterprise Level Projects
      • Creating a Project
      • Working with Tasks
      • Closing a Task
      • Closing a Project
      • Project Assistants and Work Output
    • Administration Functions
      • Extensions
      • Custom Function Setup
      • Setting up a Database
      • Input Adaptors
      • Credential Vault
      • Log Adaptors
      • Server Definitions
      • Managing Large Clusters
      • Setting up the Flight Recorder in the Console
      • Creating a Case Manager Definition
      • Work Output
      • Project Definition
      • Managing Users
      • Managing User Roles
      • Managing Access Rules
      • Authenticating via LDAP
      • Authenticating via SAML
      • Audit Log
    • Installation and Configuration
      • Installing on macOS
      • Installing on Red Hat Enterprise Linux
      • Installing on Windows - Quick Start
      • System Requirements
      • Server Installation
      • Understanding the Console Deployment Structure
      • Console Server Type Configuration
      • Console Server Configuration
      • Removing Other Unnecessary Components
      • Installing Inline with an Existing Application
      • Installing a Stand-alone Multi-Protocol Feed Server
      • Installing a Stand-alone Built in Forwarding Proxy
      • Creating a Cluster Slave Console Instance
      • Understanding the magic.properties Configuration Settings
      • Port Numbers and how to Change Them
      • Dealing with Internal Failures
      • Linking Multiple Applications Together into a Single Server
      • Monitoring by External Systems
      • Installation Considerations for Portals
      • Docker Configuration Guide
      • AWS User Deployment Guide
      • Google Cloud User Deployment Guide
    • Creating a Stand-alone Built in Forwarding Proxy
      • Installing with Ubuntu and mySQL
    • Setting up a DNS override for Data Loss Prevention
      • Deciding what to Override
      • Installing a DNS Server for the Overrides
      • Setting the Override
      • SSL Considerations
    • Backup and Restore
      • The Server Component
      • The Console Component
    • Third Party Software
      • Open Source Components
      • Closed Source Licensed Components
      • Licenses
  • PROGRAMMER'S GUIDE
    • Composable Agentic Platform Concepts
    • Setting up a Development Environment
    • Creating a Rule
    • The Kapow Extension - Example
    • Java Documentation
  • CONSOLE WIZARDS
    • Create REST with JSON service
    • Create JDBC database definitions
    • Create data set maintenance custom function
  • BEST PRACTICES
    • Naming Conventions
    • Comments & Descriptions
    • Application Configuration
    • Rules & Rule Sets
    • Repositories
    • Data Manipulation
    • Internationalization
    • Debugging
    • Performance Testing
    • Tag Replacement Techniques
    • Best Practices Repository
  • GUIDES
    • Hello, World!
    • TCL Script Writer Reference
    • Windows Automation Reference
    • Browser Certificate Installation Guide
    • Web Development Guide
    • Using the Push Notification Framework
    • Raspberry Pi with PiFace Reference
    • TomorrowX Portal User Guide
    • Examples
      • CSRF attack prevention
      • Two Factor Authentication
      • Frame Busting
      • Google Analytics
      • DNS Multi Protocol
      • Customer Satisfaction Survey
  • CHANGE HISTORY
    • New in Version 8
    • New in Version 7
    • New in Version 6
    • New in Version 5
    • New In Version 3.1
    • New In Version 3.0
  • 🌏TIME ZONES
  • ❓TROUBLESHOOTING
    • DB2 SQL error: SQLCODE: -443
    • Error 404 when trying to access the console
    • java.io.IOException: Too many open files
    • Accidental deploy to console server
  • ⚙️BROWSE COMPONENTS
  • EULSA
Powered by GitBook
On this page
  • Selecting multiple rules
  • Cut, copy and paste
  • Disconnecting chains
  • Distinguishing between variables, text and numbers
  • Don't close before testing
  • Regular expressions
  • Character ranges
  • Counting
  • Grouping and alternation
  • Positioning
  • Lists and Data Sets
  • Using the New Rules Wizard
  • Selecting or creating a repository
  • Preparing the data for the wizard
  • Creating the rule sets and configuration
  • Using the New Rules Wizard for stress testing
  • Preparing the data for the stress test
Export as PDF
  1. PRODUCT REFERENCE
  2. X Agent details

Tips and Techniques for Working with the Rules Editor

The rules editor is designed to be easy to use. There are however a few tricks that will make using the rules editor even easier. The following covers a few of these tricks.

Selecting multiple rules

If you need to move multiple rules around in unison or select them so that you can copy them to another rule set, hold down the CTRL key whilst clicking and selecting rules. Alternatively, you can hold down the mouse key and drag a rectangle around the rules you wish to select.

Cut, copy and paste

Rules can be cut, copy or pasted within the same rule set or between open rule sets by right-clicking whilst the rules are selected. To paste the rules into a new position, right-click on the canvas where the rules should be placed and select Paste.

Note: Not all browsers support the right-click feature. For this reason, edit options can also be obtained by holding down shift and left-click.

Disconnecting chains

If you mistakenly connect one rule to another, you can remove the connection by right-clicking on the chain point and selecting Disconnect.

Distinguishing between variables, text and numbers

Many of the rules in the system take a variable, text or a number as a parameter. Generally speaking, variables may only contain letters, numbers and the characters: ‘_’, ‘:’ and ‘.’, and they must start with a letter.

There will be times where a variable can be confused with a text literal, and for that reason text literals should always be enclosed in double quotes. For example: ABC is a variable whereas “ABC” is the text ABC.

Numbers on the other hand are unambiguous and can just be keyed as numbers.

When entering a CSV list of values, there is no need to enclose the entire block in double quotes. Since CSV text has commas in it, it will automatically be detected as a list of string elements.

Don't close before testing

There is no need to close a rule set to test it. You can keep multiple rule sets open in multiple windows, deploy your rules, test and then return to the already open windows to continue editing. Click the save button rather than the close button to stay on the page.

Regular expressions

Several of the standard rules make use of a string matching feature known as regular expressions. Books have been written about regular expressions and it is beyond the scope of this manual to cover more than the very basics.

In its most basic form, you can use regular expressions to see if a certain text is available within another, to count characters and to look for certain pieces of text at certain positions within words.

An example of a regular expression would be: (ab|cd)

This expression will check if a text contains the character sequence “ab” or the sequence “cd”. So both “baby” and “lcd” would be a match.

The following tables list some of the common uses of regular expressions and how they can be used to validate text.

Character ranges

Character(s)

Description

Example

Matches

Does not match

[]

Matches any of the enclosed characters

AB[cC]

ABC, ABc

abC, def

[^]

Does not match any of the enclosed characters

AB[^cC]

ABD, ABz

ABc, ABC

[a-z]

Range

AB[C-F]

ABC, ABD, ABF

ABG, ABc

.

Any character except newline

N.

N1, N2

N

\d

Any digit between 0 and 9

N\d

N1, N2

NA

\D

Any non-digit character

N\d

NA

N1, N2

\w

Any letter, digit or underscore (equal to [a-zA-Z0-9_])

ab\wd

abcd, abCd, abad

Ab/1d

\W

Any character other than a letter, digit or underscore (equal to [^a-zA-Z0-9_])

ab\Wd

Ab/1d

abcd, abCd, abad

\s

Any single white space character (space, new line etc.)

ab\sde

ab de

abde

\S

Any single non-white space character

ab\Sde

abcde

ab de

Counting

Character(s)
Description
Example
Matches
Does not match

*

Zero or more occurrences

Tes*

Test, Television, Tess

Two, Trees

?

Zero or one occurrence

Colou?r

Color, Colour

Colouur

+

One or more occurrences

Tes+

Test, Tess

Television

{x}

Exactly x occurrences of the previous character

Hel{2}o

Hello

Helo,Helllo

{x,}

x or more occurrences of the previous character

Hel{2,}o{2,}

Helllloooo

Hello, Heloo, Howdy

{x,y}

Between x and y occurrences of the previous character

Hel{2,3}o

Hello, Helllo

Helo, Hellllo

Grouping and alternation

Character(s)
Description
Example
Matches
Does not match

()

Group together

(abc){2,}

abcabc

abc

x|y

Either x or y

(abc|def){2,}

defabc, abcabc, ghjabc

abc

Positioning

Character(s)
Description
Example
Matches
Does not match

^

Match at beginning of string

^hello

hello world

say hello

$

Match at end of string

hello$

say hello

hello world

\b

Match at beginning or end of a word

\bhel

say hello, hello world

Chellos

\B

Match inside a word

\Bhel

Chellos

say hello, hello world

(?=x)

Only if followed by x

ab(?=c)

abc

ab, abd

(?!x)

Only if not followed by x

ab(?!c)

ab, abd

abc

Lists and Data Sets

Lists and data sets provide an efficient way to work with keyed data that needs to be stored either in memory or a database.

Lists are capable of storing normal variables, lists or data sets, allowing effectively for multi-dimensional arrays.

There are two types of lists: Regular lists and fixed sized lists. When you insert a value into a list, the list will automatically be created as a regular list if it hasn’t already been created.

Fixed size lists are extremely useful for memory caching. Elements inserted into a fixed size list will stay there until the maximum size of the list is reached. Once that happens, the oldest element that has not been accessed (read or updated) will be removed from the list.

Elements inserted into a list must have a fixed key. If a new element is inserted with an existing key, the existing element will be replaced.

It is possible to create global lists by creating the list in a startup rule set and then setting it as a global variable. When the global list is read from a normal rule set, any changes made to that list as a local variable will directly affect the global list. This provides a means for caching data at a global level.

Data sets provide a way to create a collection of correlated data. For example, you can define a data set called “Fruit” with the properties Name, Color and Shape. Once you have defined a data set, you can create instances of it in a database or in a list. For example: Apple, Red and Round.

The X Agent will automatically handle the correct storage of the data set in a database and properties of the data set can be added and/or removed at any time in its lifecycle. So, if at a later stage you need to store another property in your Fruit data set, you can simply add it to the definition.

Data sets should be defined in a start-up rule set and can only be defined once within the life cycle of a deployment.

Data sets can optionally contain a number of lists. However, lists stored within a data set may only be single dimensional (you cannot have lists within lists).

Once a data set is stored within a database or a list, you can read it by key, delete it, update it etc. To update a data set within a list, you simply create it again with the same key name.

Using the New Rules Wizard

A quick way to get started building rules for a new web application or for stress testing is to use the rules wizard. The rules wizard uses live test data to extract URIs visited and build a structured collection of rule sets.

Selecting or creating a repository

The rules wizard creates a large number of files, so we strongly recommend that you create a new repository to write the new files to.

Preparing the data for the wizard

Before using the rules wizard, you must first deploy and start either the RuleWizardConf or the StressTestConf configuration found in the Rules Wizard repository to a test server that is protecting the target application (alternatively you can use the zero installation test method which is covered later in the manual). Once done, simply begin navigating the various components of the target web application.

Once you have visited all of the pages you wish to cover with your initial rule set, return to the console and go to the server status screen:

In this example the Qwerty demo application was chosen and there are 19 test records ready to be processed.

Creating the rule sets and configuration

Now click on the "New rules wizard" button. You will be presented with the following page that controls the wizard:

At this point you can have the rules wizard create a filter for the rules that automatically exclude static content. It is a comma separated list and you can easily add new elements.

Make sure that you select either New rule set or Stress test, depending on your requirements.

When you have selected an appropriate repository (in our case, we have use the repository ‘Name’) and reviewed the exclusion list, click on "Create".

After a brief pause, the X Agent will write out a complete configuration and collection of rules. The following pages show how these rule sets are structured for a new rule set:

Understanding the Load page

In keeping with best practice for rules writing, the rules wizard always creates a "Load" rule set. This contains rules that are generic for all URIs. The name of the page will be the name of the repository followed by the word "Load".

This rule set will first check for malformed HTTP requests and, if found, will reject them. Subsequently it filters out static content, adds a tracker rule and then proceeds to the main rule set.

Understanding the Main page

The main rule set page contains a basic structure that determines the URI being visited and then uses a switch to re-direct to a rule set covering that URI.

Each one of those rule sets, in turn, are blank, but are already created and ready to have rules added to them. The new rules wizard creates a quick foundation to get you started with writing rules for your application.

Using the page hints

When the rules wizard creates each of the blank templates it includes sample information of the fields gathered and their values in the page description:

You can use this information to identify the fields available to your rules.

Using the New Rules Wizard for stress testing

As described above, an alternative use of the New Rules Wizard is to create a set of rules for stress testing an application. To do this, you must first deploy the StressTestConf configuration from the Rule Wizard repository to the web application on the browser proxy.

Preparing the data for the stress test

Once deployed, you should work through the application to be stress tested, step by step. Try to complete pages as normally as possible, making sure not to pause unnecessarily during the process, as wait times are recorded.

During the new rules wizard creation, select Stress test instead of New rule set.

The following pages show how the rule sets are structured for a stress test:

Understanding the Load page

In keeping with best practice for rules writing, the rules wizard always creates a "Load" rule set. This contains rules that are generic for all URIs. The name of the page will be the name of the repository followed by the word "Load".

This rule set allows you to set up a multitude of things. First of all, the target server identity, and also the user agent to use. It is very common to modify the Load rule set to pick up randomized values for elements such as users. The following shows a modified Load rule set to include diversified User ID and Password configurations for a Qwerty stress test:

In this scenario, the users and their passwords are picked from a CSV file of valid entries. The settings for the CSV Line Picker rule are as follows:

It is important to notice how the THREAD_NO variable is used to pick a line in the CSV file. The THREAD_NO is incremented for every stress testing agent and should the number exceed the number of entries in the CSV file, the rule will start reading from the beginning again.

Understanding the Main page

The main rule set page contains a basic structure that determines the URI being visited and then uses a Number Sequencer rule to re-direct to a rule set covering that URI.

Understanding the Wait page

The wait rule set page is used to define wait times. This delay can be fixed, or it can be a function of the recorded wait time during rule set creation.

The following shows a wait time delay:

The wait time properties can be set as follows:

The default for the percentage random is zero, but it can be randomized to create a more realistic user load.

An alternative is to either remove the wait time completely or insert a fixed delay:

Understanding the Fail page

The fail page is used to define any action to be taken if a stress test page invocation fails. The default is to abort the flow and stop the thread:

Understanding each stress page

Each page that is detected will be assigned a unique page name and will be given a unique rule set. The following shows the main page in the Qwerty application:

In this case, the page is a GET and the invocation is relatively simple. For POST requests, the structure looks slightly different:

In this case, the recorded POST variables are set up in a single rule. You can override these POST variables to use pseudo random values (as shown previously in the Load rule set).

The individual pages are where you should include test data, and possibly review the response data from significant pages to ensure that the stress test is progressing well.

PreviousRule SetsNextWeb Application Rule Set Patterns

Last updated 4 months ago

Qwerty Demo Server
Create new rules wizard
NameLoad rule set
NameMain rule set
Rule infos tab
NameLoad rule set
Modified rule set for Load rule set
props
NameMain rule set
NameWait rule set
NameWait props
NameWait rule set
NameFail rule set
Name_main rule set