LogoLogo
  • Documentation
  • PRODUCT REFERENCE
    • Introduction
    • Concepts and Terminology
      • Console
      • Servers (X Agents)
      • Projects
      • Configurations
      • Rules Editor
      • Rules, rule catalogue, protocol rules and rule sets
      • Test Data
      • Trace Data
      • Flight Recorders
      • Case Managers
      • Data Files
      • Content Files
      • Performance Data
      • Extensions
      • Protocols
      • Credential Vault
      • Custom Functions
      • Databases
      • Input Adaptors
      • Users
      • User Roles
      • Access Rules
      • Repositories
      • Audit Log
      • Proxies
    • Architectural Scenarios
      • Command and Control
      • Simplest Form
      • Servlet Filter
      • API Transformation
      • Active Web Proxy
      • Web Application Server
      • Active Proxy With Content
      • Mobile Application Server
      • Asynchronous Multi-Protocol
      • Data Loss Prevention Architecture
    • Getting Started
      • Login
      • Essential Things to do First
      • Keeping the Product Current
      • Common Console Management Tasks
      • Viewing Active Servers
    • Quick Product Introduction
      • Preparing the Browser Proxy
      • Setting up the Proxy in the Browser
      • Verifying the Browser Configuration
      • Understanding the Configuration
      • Understanding input and variables
      • Preparing a new repository
      • Locating the Page to Modify
      • Determining the Actions Required
      • Building the First Rule Set
      • Setting Rule Properties
      • Connecting up the First Rule
      • Getting a Server Result
      • Manipulating the Server Result
      • Returning the Result to the User
      • Creating a Configuration for the Rule Set
      • Selecting the Input Source
      • Deploying the New Configuration
      • Testing the Rules
    • X Agent details
      • Configuration Settings
      • Rule Sets
      • Tips and Techniques for Working with the Rules Editor
      • Web Application Rule Set Patterns
      • Building Rule Sets for Inclusion
      • Notes on Working with SOAP
      • Zero Installation Rules Testing
    • Data Files
    • Content Files
      • Previewing Content Files
      • Editing Content Files
      • Deploying Content Files
    • Test Data
      • Uploading and Downloading Test Data
    • Performance Data
      • Setting the Performance Collection Level
      • Retrieving the Performance Data
      • Viewing Transaction Counts
      • Viewing Transaction Counts and Inline Time
      • Viewing the Complete Performance Report
      • Understanding the URI Performance Information
      • Performance Benchmarks
    • Trace Data
      • Understanding the Trace
    • Tracing the Result of a System Failure
    • Conjoined Performance and Trace data
    • Live Performance Data and Probes
      • Setting Probes
      • Live Performance Impact Considerations
    • Accessing Server Logs
      • Viewing a log
    • Working with Flight Recorders
      • Searching Flight Recorder Information
      • Retrieving Flight Recorder Data
      • Graphing Flight Recorder Data
    • Working with Case Managers
      • Case Manager Definitions
      • Navigating the Case Managers
      • Creating a Case From Rules
      • Creating a Case From the Console
      • Viewing the Queues
      • Picking a Task
      • Case View
      • Completing a Task
      • Task History
      • Searching Tasks
      • Searching Cases
    • Managing Repositories
      • Moving Files Between Repositories
      • Copying Configurations and Rule Sets Together
      • Copying Dependent Rule Sets
      • Backing up
      • Moving Entire Repositories Between Consoles
      • Setting Repository Imports
      • Specifying Repository Rule Group Restrictions
    • Version Control and Restoring Files
    • Using the Portal UI
      • Using the Start Menu
      • Positioning Windows
      • Resizing Windows
      • Maximize, Restore and Minimize
      • Storing and Switching Desktops
    • Enterprise Level Projects
      • Creating a Project
      • Working with Tasks
      • Closing a Task
      • Closing a Project
      • Project Assistants and Work Output
    • Administration Functions
      • Extensions
      • Custom Function Setup
      • Setting up a Database
      • Input Adaptors
      • Credential Vault
      • Log Adaptors
      • Server Definitions
      • Managing Large Clusters
      • Setting up the Flight Recorder in the Console
      • Creating a Case Manager Definition
      • Work Output
      • Project Definition
      • Managing Users
      • Managing User Roles
      • Managing Access Rules
      • Authenticating via LDAP
      • Authenticating via SAML
      • Audit Log
    • Installation and Configuration
      • Installing on macOS
      • Installing on Red Hat Enterprise Linux
      • Installing on Windows - Quick Start
      • System Requirements
      • Server Installation
      • Understanding the Console Deployment Structure
      • Console Server Type Configuration
      • Console Server Configuration
      • Removing Other Unnecessary Components
      • Installing Inline with an Existing Application
      • Installing a Stand-alone Multi-Protocol Feed Server
      • Installing a Stand-alone Built in Forwarding Proxy
      • Creating a Cluster Slave Console Instance
      • Understanding the magic.properties Configuration Settings
      • Port Numbers and how to Change Them
      • Dealing with Internal Failures
      • Linking Multiple Applications Together into a Single Server
      • Monitoring by External Systems
      • Installation Considerations for Portals
      • Docker Configuration Guide
      • AWS User Deployment Guide
      • Google Cloud User Deployment Guide
    • Creating a Stand-alone Built in Forwarding Proxy
      • Installing with Ubuntu and mySQL
    • Setting up a DNS override for Data Loss Prevention
      • Deciding what to Override
      • Installing a DNS Server for the Overrides
      • Setting the Override
      • SSL Considerations
    • Backup and Restore
      • The Server Component
      • The Console Component
    • Third Party Software
      • Open Source Components
      • Closed Source Licensed Components
      • Licenses
  • PROGRAMMER'S GUIDE
    • Composable Agentic Platform Concepts
    • Setting up a Development Environment
    • Creating a Rule
    • The Kapow Extension - Example
    • Java Documentation
  • CONSOLE WIZARDS
    • Create REST with JSON service
    • Create JDBC database definitions
    • Create data set maintenance custom function
  • BEST PRACTICES
    • Naming Conventions
    • Comments & Descriptions
    • Application Configuration
    • Rules & Rule Sets
    • Repositories
    • Data Manipulation
    • Internationalization
    • Debugging
    • Performance Testing
    • Tag Replacement Techniques
    • Best Practices Repository
  • GUIDES
    • Hello, World!
    • TCL Script Writer Reference
    • Windows Automation Reference
    • Browser Certificate Installation Guide
    • Web Development Guide
    • Using the Push Notification Framework
    • Raspberry Pi with PiFace Reference
    • TomorrowX Portal User Guide
    • Examples
      • CSRF attack prevention
      • Two Factor Authentication
      • Frame Busting
      • Google Analytics
      • DNS Multi Protocol
      • Customer Satisfaction Survey
  • CHANGE HISTORY
    • New in Version 8
    • New in Version 7
    • New in Version 6
    • New in Version 5
    • New In Version 3.1
    • New In Version 3.0
  • 🌏TIME ZONES
  • ❓TROUBLESHOOTING
    • DB2 SQL error: SQLCODE: -443
    • Error 404 when trying to access the console
    • java.io.IOException: Too many open files
    • Accidental deploy to console server
  • ⚙️BROWSE COMPONENTS
  • EULSA
Powered by GitBook
On this page
Export as PDF
  1. PRODUCT REFERENCE

Working with Flight Recorders

PreviousViewing a logNextSearching Flight Recorder Information

Last updated 1 year ago

Flight recorders are essentially on-demand log files. Their most common use is to gather data for later study or forensic requirements.

A good example of the use of a Flight Recorder is the following snippet adapted from a banking demo:

In the above example, all of the data about the particular access to the application has been obtained with the HTTP Request Tracker. The next rule is an Injection Checker rule that will check all submitted fields for Java Script, IFRAMEs, SQL injections etc. Should one be found, a Flight Recorder Trigger is invoked, recording the field in which the attack is found, the contents and the URL of where the attack occurred. This data is keyed by DEVICE_ID in this case. The full properties for the Flight Recorder Trigger are shown below:

When this rule is hit, a number of things happen:

An open record is written to an index table (whose name will start with “ATTACK” and which will be found in the “HISTORY” database. Should the table not exist, it will be created). A record of all the fields presently available for rules is also stored in a data table (using the same creation conditions as the index table).

Up to 5 user-defined index fields are also written to the index table, alongside a number of default data elements, such as the browser type, IP Address and User ID (if known).

So now we have a complete record of what happened at this specific point in time. We then close the rule set by logging the user off (since attempting an attack is a pretty serious violation of terms of use).

However, we may wish to continue recording any further activity performed by this user, since it may yield useful forensic information. So, we also insert a Flight Recorder Add rule.

The index table is keyed by DEVICE_ID, meaning that subsequently, if the same DEVICE_ID is found by a Flight Recorder Add rule, it will add another record of all of the fields presently available to the Flight Recorder’s data table. The properties for the Flight Recorder Add rule are as follows:

This essentially maps the key fields to the trigger.

Now, we not only have the data from the point in time of the attack, but also, we have started a specific log of all subsequent activity coming from that DEVICE_ID.

This data is recorded in the same format as test data and can be downloaded and used in the same way (as a Microsoft Excel compatible spreadsheet or by running rules over it on the test server).

Searching Flight Recorder Information

Retrieving Flight Recorder Data

Graphing Flight Recorder Data

banking demo with Flight Recorder
Flight Recorder Trigger props
Flight Recorder Trigger props