If upgrading from a previous version, here are some of the highlights and new features.
The following is a list of new features that directly affect most users and administrators.
Support for Input/Output variables definitions for rule sets. This enables included rule sets to work with variable names in isolation from its parent rule set.
You can now specify which rule groups can be used by the rules editor in a given repository. This allows for the segmentation between basic and advanced users.
The console now allows the administrator to assign one of two UIs (Classic or Portal) to a user. Users in portal mode will get a more limited view of options, but also a better ability to customize and save their preferred desktops.
The console now allows a single console to contain all extensions separated by version, the definition of a version against a repository and can control the deployment of rules so that the correct version of extensions are deployed only to servers that support that extension.
Version 8 rules engines are now capable of registering their own protocol support and can deploy network listeners on one or more network ports at start-up time. In addition, a new protocol analysis and breakdown feature has been added to facilitate rules writing against arbitrary network protocols. This is known as Outerware. Network transport supported includes TCP, SSL and UDP.
A new multi-protocol input adaptor has been introduced to assist with managing non-HTTP protocols.
The rules engine now supports "chatty" protocols (such as SMTP). This enables OctopusV8 support for protocols that have multiple interactions on the same network stream before an actual transaction occurs.
Constants provided to any rule may now include special characters escaped using the Java escape syntax ("\r" = CR, "\n"= LF etc.).
A number of new rules have been added including list management rules, stress testing rules and rules to deal specifically with protocols.
Repositories can now be closed so that they no longer appear in the administration tree. This is an alternative to deleting repositories, as closed repositories can be re-opened with a single click
A new Rules Wizard configuration and a new stress testing input adaptor allow for the capture of the flow of a web application and the play back of that flow in many threads, including customization with multiple users, semi-random wait times, ramp up time and the full set of rules to validate output.
It is now possible to switch modes (starting point rule sets), and set global variables specified in configurations, whilst a configuration is deployed and running.
When working with a large number of repositories and servers within a single console, this feature allows the user to cut down the number of visible repositories and servers at any given time.
The product now allows a single user to have more than one role.
A new debug mode is available for all input adapters. This replaces the previous debug mode used in test input adapters. The most significant use of this feature is the ability to make Exit rules list all variables in debug mode only. Exiting rules relying on test debug mode have also been updated to use the universal mode.
New built in project wizards can assist in creating project template with To Do lists for all parties involved and call also keep project sponsors and owners updated on the progress of a project.
The following is a list of major changes that are relevant to administrators and extension programmers.
The browser proxy now provides the ability to create on-the-fly valid certificates so that sites with HTTP Strict Transport Security still functions through the browser proxy and shows up as having a valid certificate. After installing the latest BIP Runtime extension, please refer to the Browser Certificate Installation Guide in the Certificates folder.
A new secondary rules engine designed specifically for breaking down network protocols has been introduced. This rules engine is only configurable by administrators, and even though extended through the normal extension process, carries a completely new programming model to facilitate the more technical aspect of protocol disassembly and assembly.
This new server type is specifically designed to handle implementations of OctopusV8 that are living outside an application server. It is effectively an extension of the feed server used in earlier versions.
Multi-protocol servers are capable of running and proxying SSL based protocols using the traditional Tomcat crt & key certificate files combination.
The VAO object now supports the use of keyed lists that can be accessed via key, enumerated, sorted and manipulated with add/delete/update.
Data sets represent a collection of variables that can be located with a key. Data sets are capable of being created into databases or into lists.
The update server can now be accessed using the user browser as a relay via cross-origin resource sharing. This allows consoles with no internet access to follow the usual update path.
Rule sets will now be locked to the first user that opens a rule set for editing. Users trying to access an already opened rule set will receive a notification that the rule set is locked and the save option will be disabled until an exclusive lock can be obtained.
Extensions are now provided to interact with LEDs, relays on switches using Raspberry Pi and the PiFace Digital board.
New wizards can now be delivered via extensions in the console UI. This allows wizards for a wide range of assistance functions in the console to be dynamically plugged in.
This document outlines changes since the official version 3.0 release.
This new input adaptor allows for the processing of structured data dumps from systems such as mainframes.
Bad configuration of data base settings now results in a meaningful message being shown in the console.
A new type of plug-in allows the administrator to configure additional logon requirements, such as a second factor or network access restrictions. In this release only a local host access only and email second factor plug-in is shipped. However, this plug-in structure enables logon extensions such as single sign-on, LDAP, tokens, Tivoli Access Manager etc., to be used for the console. These plug-ins can be custom written on demand.
A new configuration setting (preserve stream) and a setup template that illustrates how the filter can be used to protect PHP applications has been added.
This new console extension allows for configurations, rule sets and deployable data to be logically separated amongst users. Specifically this allows repositories to be created that are unique to a given task or workflow (such as Test, Staging and Production). Different individuals can be assigned different permissions (through the user roles) to determine if they can view, update and deploy from a given repository. When coupled with server permissions this allows complete control over who edits, checks and deploys rules to any given server.
Any changes to configurations, rule sets or data are now automatically backed up for perpetuity. This includes the deletion of any given file that is linked to a repository. Older versions of files can be restored by a simple point in time selection.
Files can be seamlessly copied between repositories of the same kind (data to data, rule sets to rule sets and so on). To make it simpler to administrate large collections of rule sets that belong to a given configuration, copying of configurations also allows the automated drag-along of all the rule sets the configuration requires to function. Optionally this drag-along can be preceded by the clearing of the target rule set repository. Copying that override or clears target files results in an automated backup or the target files first.
The HTTP Session Attributes rule is designed to be used during exploration of a new application. It will list the contents of anything stored in the application session during the flow of the application. This rule is not for production use.
The Language Hash rule provides a phonetic simplification of any word provided to it, so that it can be used to compare to other words. This is commonly used to catch basic misspelling of names and addresses..
The MD5 Hash rule can take any textual input and convert it to a one way encryption using an RSA compliant MD5 algorithm. This is mainly used to encrypt credit cards numbers when they need to be used for future reference in a data base. The resulting hash can not be decrypted, but instead an encrypted version of another provided text can be used to verify equality.
Allow enter key to be used for logon Add TOC to documentation PDF On copy, make sure the newly copied rules is selected On delete, make sure the parent id of the deleted item is selected Do not allow main frame to be broken out from logo header
– none –
New rules added (MD5 Hash, HTTP Session Attributes, Language Hash) Access Rule Configuration PHP Setup and details Repositories Repository security additions for users
The console server and universal appliance have been consolidated into a single deployment. Only one installation will be shipped. If a feature is not desired from the deployment it can be deleted.
A console file can now be renamed (configuration, rule set etc.), by giving it a new name and saving it.
A new rules wizard now helps to build an initial collection of rule sets and a matching configuration based on collected test data.
The console itself is now protected with OctopusV8 and in addition it is now possible to add new features to the console itself as custom functions. An example showing how to maintain an IP address block list and how to add additional password rules is included in the default deployment.
Any built in forwarding proxy (forwarder) can now be configured from the console instead of using the web.xml file. This configuration is performed at server level.
The server now ships with a built in forwarding proxy and web proxy deployed by default. Both the appliance and web proxy are available from the local host only unless configured otherwise.
The addition of a web proxy allows for easy initial testing of rules for any given website without any additional installation. To test rules against any site anywhere, simply configure the browser used to access the web proxy of the built in forwarding proxy.
It is now possible to redirect an incoming request to a different server or host, based on the virtual host
The ability to redirect the virtual host of the incoming request now allows the installation of the built in forwarding proxy on the same server as the site being protected. This requires port changes to the existing application, but is otherwise transparent.
It is now possible to create an alias for a given database name. This allows stock rules to be written using a generic database name that can be aliased using the configuration.
It is now possible to set database pool size limits, pool full conditions and other database level parameters.
The console now contains a security audit log that contains an audit trail for all major changes to administrative items.
Extension and user roles are now automatically backed up when changes occur and can be restored using the normal restore features.
Two new user types have been added: Super User and User Administrator. Super users can perform all administrative functions except user maintenance. User administrators can only perform user maintenance. This change was made to facilitate workflow inside large organizations.
It is now possible to set a performance monitoring level in the configuration (Input Source). The options are to just collect transaction counts, collect transaction counts and timings (giving individual rule delays) or collect transaction counts, timing and individual URI response times. A comprehensive reporting framework has been added to analyze the results.
It is now possible to automatically document the entire extent of a configuration deployed to a given server. Documentation includes server information, configuration settings, rule sets, data files, content files, JDBC driver settings and more.
The demo console can now be configured and restructured to one of four different production scenarios directly from with the console itself. Many settings can also be changed without requiring a restart.
The following is a list of new/changed environments supported since version 5.0.
From version 6, non-Jetty installations are no longer supported. The server must be deployed using Jetty. To most users this is a transparent issue and does not affect functionality in any way.
This feature allows a single OctopusV8 configuration to be deployed to more than one server on the same piece of hardware. The rules engine detects the conflict and does an automatic resolution by selecting an alternate management port and moving its home folder to a sub-directory below the main home folder.
As of version 6, there will only be one download and installation image covering all OctopusV8 installation variations. The Universal appliance and feed server equivalents can be created using this one image.
The built in proxy (formerly Universal Appliance) now ships as an extension (BIP Runtime) allowing the entire proxy to be updated using the update server.
JNDI data sources can now be used to access databases defined within the target application server's JNDI name space.
Custom logging (other than System Out) is now supported. Logging options include system out logging, file logging and logging through Apache commons logging (which supports Log4J, Avalon LogKit and basic JDK 1.4 logging).
If upgrading from or have had exposure to a previous version, here are some of the highlights that were introduced in version 7.
The following is a list of new features that directly affect most users and administrators:
Integration with Amazon SES SMTP and support for the Amazon Elastic Load Balancer provides for complete cloud-based installations. Please note that for Amazon SES, even though the Amazon documentation calls for TLS, they do in fact mean SSL and do not support StartTLS. This is being highlighted here as it can be very confusing.
The documentation has been split into guides that are more relevant for specific users. This cuts down the size of reading materials and allows for the specific documents to be better targeted. In addition, the rules reference is now a separate document that is built upon demand, based on the actual rules installed in the console.
The new credentials vault provides a central place to store user IDs, passwords, access keys and the like. This ensures that this sensitive information no longer needs to be stored directly within the rules in clear text. Instead, the relevant credentials are supplied to the rules engine in encrypted form as part of the deployment process.
The spy cam feature provides a method for recording and visually playing back user interactions with a website. This feature is currently experimental and has a significant number of limitations. It is included in this version as a technology preview and has limited support.
The rules editor has been given a face lift with newer larger connection points, better graphics and an editing grid.
Within the rules editor, any rule on the canvas can now be right-clicked to access the help documentation on that specific rule which was previously only found in the full product reference document.
Test data is no longer collected by default. With the introduction of the built in browser proxy in version 6 there has been a steady decline in the need for the test server. Instead, there is now the ability to turn test data collection on and off on demand, providing a better balance between memory consumption and actual data needs.
User roles can now be used to grant/restrict access to specific flight recorders.
A number of new rules have been added (including the Filter rule and experimental rules to support HBase). The HTTP Servlet Execute rule has been renamed to HTTP Server Execute to make it more intuitive.
Due to the large increase in rules over the years, the rules editor has now been equipped with a search feature that includes instant results based on search characters typed.
The user interface has been given a face lift which includes new graphics, more consistent button icons and a better login experience.
Console users can now be authenticated against a central LDAP server
The following is a list of major changes that are relevant to administrators and extension programmers.
By moving the various user IDs, access keys and passwords used by rules to access external services to the credentials vault, those items are no longer exposed to the rule writers and generic rule sets can be shared without the risk of also accidentally sharing restricted information. In addition, those credentials are now only transmitted to the rules engine in encrypted form.
To facilitate the use of resource bundles and properties files for internationalization of rules engine output, the home folder of the rules engine is now included in the class path of the rules engine loader. This inclusion applies to individual files only, not JAR files.
All extensions now have the ability to include code that will be run upon installation and removal. This allows extensions to automatically create credential templates in the credential vault, install database drivers or perform other console related tasks as part of the install.
The console now makes use of a technology known as COOP (Change Oriented Object Persistence). This technology significantly reduces the time needed to add new features to the console. The database will now automatically adapt to the configuration objects instead of the other way around. There is a one-time conversion step where data is copied from the old table structure to COOP when first migrating from version 6 to version 7, but that will be the last database update ever needed for console configuration items.
Rules extensions now ship with the ability to include rule documentation. The rule documentation is automatically merged into the rules reference PDF document as part of the install process. It is also made available as a help feature in the rules editor. Please refer to the Programmer's Guide for more information.
Emails sent from both the rules engine and the console now support both SSL and StartTLS (Transport Layer Security with fall back) for SMTP emails.
The console can now be remotely managed by providing scripts in the TCL language via HTTP.
The user interface has been given a complete makeover to improve usability.
Before a configuration is deployed to the server it is now checked for file reference integrity to avoid deployment mistakes.
Entire repositories can now be packaged up for distribution between consoles and can be progressively backed up.
Targeted logs can now be added that will continue to monitor a user, IP Address, Device ID or some other provided identifier and log test data whenever that identifier is present. This data can be used for forensic analysis or to monitor specific data points (such as which browser type users are using).
Flight recorder data can be easily graphed and visualized.
A very flexible case manager that integrates fully with the Flight Recorder technology has been added. It provides customizable workflow and event management abilities.
The ability to trace up to 20 transactions through the rules engine has been added. This allows for a precise text view of how a rule set executes with any given transaction.
Rules are now available to help protect against XSS, CSRF, XPATH, IFRAME and SQL Injection attacks.
A license key is now supplied for each installation to determine the licensor and the features licensed.
A new rules class loader has been introduced to load classes from the product home folder instead of the application path. This includes all third party JAR files so that conflict resolution between application level JAR files and rules JAR files are no longer required.
The basic application level rules engine (JAR) now loads a minimum of classes to allow as many features as possible to be delivered via extensions. Extensions can now be changed/added without the requirement of the web application being restarted, even after code has already been loaded.
The connection between the console and the rules engine can now be encrypted for better security.
Logged events are now more consistently time stamped and include a severity determination to help with diagnostics.
The version control utility now covers all elements of repositories (including data) for consistency.
Please see the rules reference for a complete listing of rules and the version in which they have been added. More than 30 new rules have been added since version 4.0.
The console and demo server are now able to obtain a list of updates that can be downloaded and applied on demand. Updates include sample applications, extensions and sample repositories. What can be download will depend on the license held.
In previous versions of the rules engine it was possible to have a very high performance rule set with substantial console output create a race condition where the administration console output would be challenged to keep up with the flood of data. As of version 5, this condition results in the console dropping packets instead of trying to obtain a perfect list. This helps maintain the high performance and memory consumption of the rules engine.
It is now possible to nominate rule sets that will only be executed once on either startup or shutdown. This allows rules to sign in to a service, add timing code, perform cleanup and perform other tasks that may be needed to encapsulate the main rule set.
The rules engine now features global variables that can be used to communicate information between separate transactions (for example credentials and timing information).
The rules engine can now be configured to a specific event count to determine if the engine is looping. This allows the ability to ensure that very large XML files with substantial processing can be executed without a potential loop being detected.
Rule sets can now be set to be executed at a regular interval. This allows for background services to be created.
This feature allows for the creation of content (html, xml, images) directly within the console and for this content to be served up directly from within the rules engine. The content is an effective overlay of the web server content the rules engine is protecting.
The console now allows for the direct editing of HTML documents being served up by the content delivery mechanism. Further editors for other content is planned.
Content delivered by the rules engine can now be managed with dedicated rule sets without the need for file readers and dedicated HTTP responses.
The rules engine will make dynamic performance optimization decisions based on the content of the variables being used and the scope within which they are being used. This optimization can result in substantial performance gains, especially with larger variables (such as entire HTML pages).
In addition to the support in previous versions for simply banning variable names from ever being entered into the rules engine, it is now also possible to allow banned variables to appear as MD5 hashes or PCI compliant masked credit card numbers (First 6 and last 4 digits visible).
It is now possible to set variables internally as objects from within rules. This can be done both locally and globally. Objects that are set this way must be read as objects as well, also from within rules. If they are read using the normal access rules (such as Get Variable) a string representation (toString) will be returned. It is the responsibility of the programmer writing custom rules to exploit these features and to ensure the stored Objects can be released from memory and that they clear up any resources used. For more information, see the VariableAttributeObject and RulesEngine JavaDoc.
The console now contains status messages, such as when the rules engine was loaded, started, stopped and unloaded. This is especially useful to understand at what point the startup and shutdown rule sets are invoked.
It is now much easier to control and manipulate the content of HTTP requests going to the actual application (including overriding user supplied parameters) and equally it has been made much easier to modify the application response before it is returned to the end user.
The following is a list of new environments supported since version 4.0.
A new Linux based appliance provides easy support for all non-Java environments.
Array parameters provided in web requests are now correctly available to the rules engine.
The popular Struts framework has the ability to create variable names that essentially look like function calls and have previously been hard to access as the rules engine would interpret them as strings.
Test data can now be downloaded in XLS format, can be modified and uploaded as an XLS file, which in turn will be automatically converted to a .TST file (the new extension for test data files).
Large server clusters (anything from 10+ servers running the same rule set) can now be managed by a cluster of OctopusV8 consoles. Rule sets from a single master console can be automatically propagated to any number of slave consoles for further propagation to target servers. Each slave cluster can have their rules engines started and stopped as a single unit.
For rules engines and consoles that must access the internet via an internal web proxy, NTLM authentication (for Microsoft ISA servers) has been added. Additionally, rules that previously exposed the proxy settings for an external web request, now obtain their proxy information from the server settings instead.
Keyed arrays can now be created and used within a rule set. Arrays are stored as JSON objects and can be manipulated with general purpose rules. Rules for conversion between arrays and CSV formats have been added and keyed arrays are now used for HTTP header manipulation.
This document outlines changes since the official version 2.1 release.
If a user attempts more than 3 invalid logins, the user account is automatically suspended and will require a reset.
This new addition to configurations allow for a set of default values to be set for each start of the rules engine. Default values for empty columns in a CSV file can also be set.
FMT Octopus now supports the handling of Portlet requests by wrapping each individual Portlet in the same was as a normal web page would be wrapped by the inline filter.
The inline filter automatically detects the presence of XML in the incoming request and decomposes the individual XML tags into more friendly variable names.
This addition provides console support for double-byte character sets such as Chinese, Korean and Greek.
This addition provides test data support for double-byte character sets such as Chinese, Korean and Greek.
By allowing a relative path for the home folder it has become easier to create packaged stand-alone instances, such as the servers listed below.
This server is a low-volume, ready to run from USB drive demo server that includes the Console, the MyBank demo and a test server.
This server is a fully replicable stand alone server instance that can be installed multiple times on the same server, in such a way that each instance can process data simultaneously. This server forms the basis for the FMT Octopus offline appliance.
This server contains the console application and the basic deployment components needed to perform a technical compatibility exercise. It can be launched from a USB drive to save time installing any software other than the inline filter.
The product now collects a performance count for each “dead-end” or un-connected chain points on a rule, so it is possible to see how many times the dead-end has been reached.
Upload and download capabilities have been added for rules configurations.
The HTTP Session Tracker rule now has the ability to turn cookies off if required.
To improve memory consumption, empty variables will no longer be carried around on the data packet and will not be stored in test data.
The ability to send an email to an administrator in the case of an internal failure in the rules engine has been added.
It is now possible to configure the inline filter to survive an internal failure and auto-recover.
For performance reasons, it may not always be desirable to automatically output a copy of the information sent to the console to the application server’s system console. Therefore this feature has now been made optional.
In the past error messages on the server side would simply output the error code. This has been improved to now include properly formatted error messages in the default rules language.
The Alias rule is used with SOA and AJAX data to simplify the rule writing for tags in deep XML documents.
The Crash rule forces an internal failure in the rules engine. It is used for testing the email alerting feature of the product.
The FailSafe rule is capable of detecting an internal failure an make a decision on how to proceed. It can also ensure recovery if desired.
The SMTP Email rule is capable of sending an email to any number of recipients using the SMTP server defined for crash alerting.
The HTTP Header Reader is capable of reading any header encapsulated in a HTTP request. This is mainly useful to read information added by proxy servers.
Added icons in front of links on main page Created icon buttons (picture and text) for all major tasks Allowed sliding/dragable console window sizes Changed “save rules” and “regular expressions tool” into icons Added cut, copy and paste icons to rules editor Changed green dot in rules editor into something more “rounded” Masked JDBC driver password
Rules engine menu missing if extension not installed for rule set that use it. Value Comparer makes comparing to a Variable incorrect under certain conditions. Fix spelling error in rules manifest. Ignore rules sets with no “first rule”. Fix the return of a null parameter from a request that has been stored in the session Allow spaces in deployed server path Make sure that the portlet API is optional for a deployment Under certain conditions filter did not fail open
Include information on filter jar file and using shared lib for rules engine Portal configuration documentation Documentation of all new rules Documentation of SOA and AJAX Document test page creation (for “pinging”) Documentation on setting up SMTP server Documentation section on failure handling Added extension files information to rules reference Added group information to rules reference