LogoLogo
  • Documentation
  • PRODUCT REFERENCE
    • Introduction
    • Concepts and Terminology
      • Console
      • Servers (X Agents)
      • Projects
      • Configurations
      • Rules Editor
      • Rules, rule catalogue, protocol rules and rule sets
      • Test Data
      • Trace Data
      • Flight Recorders
      • Case Managers
      • Data Files
      • Content Files
      • Performance Data
      • Extensions
      • Protocols
      • Credential Vault
      • Custom Functions
      • Databases
      • Input Adaptors
      • Users
      • User Roles
      • Access Rules
      • Repositories
      • Audit Log
      • Proxies
    • Architectural Scenarios
      • Command and Control
      • Simplest Form
      • Servlet Filter
      • API Transformation
      • Active Web Proxy
      • Web Application Server
      • Active Proxy With Content
      • Mobile Application Server
      • Asynchronous Multi-Protocol
      • Data Loss Prevention Architecture
    • Getting Started
      • Login
      • Essential Things to do First
      • Keeping the Product Current
      • Common Console Management Tasks
      • Viewing Active Servers
    • Quick Product Introduction
      • Preparing the Browser Proxy
      • Setting up the Proxy in the Browser
      • Verifying the Browser Configuration
      • Understanding the Configuration
      • Understanding input and variables
      • Preparing a new repository
      • Locating the Page to Modify
      • Determining the Actions Required
      • Building the First Rule Set
      • Setting Rule Properties
      • Connecting up the First Rule
      • Getting a Server Result
      • Manipulating the Server Result
      • Returning the Result to the User
      • Creating a Configuration for the Rule Set
      • Selecting the Input Source
      • Deploying the New Configuration
      • Testing the Rules
    • X Agent details
      • Configuration Settings
      • Rule Sets
      • Tips and Techniques for Working with the Rules Editor
      • Web Application Rule Set Patterns
      • Building Rule Sets for Inclusion
      • Notes on Working with SOAP
      • Zero Installation Rules Testing
    • Data Files
    • Content Files
      • Previewing Content Files
      • Editing Content Files
      • Deploying Content Files
    • Test Data
      • Uploading and Downloading Test Data
    • Performance Data
      • Setting the Performance Collection Level
      • Retrieving the Performance Data
      • Viewing Transaction Counts
      • Viewing Transaction Counts and Inline Time
      • Viewing the Complete Performance Report
      • Understanding the URI Performance Information
      • Performance Benchmarks
    • Trace Data
      • Understanding the Trace
    • Tracing the Result of a System Failure
    • Conjoined Performance and Trace data
    • Live Performance Data and Probes
      • Setting Probes
      • Live Performance Impact Considerations
    • Accessing Server Logs
      • Viewing a log
    • Working with Flight Recorders
      • Searching Flight Recorder Information
      • Retrieving Flight Recorder Data
      • Graphing Flight Recorder Data
    • Working with Case Managers
      • Case Manager Definitions
      • Navigating the Case Managers
      • Creating a Case From Rules
      • Creating a Case From the Console
      • Viewing the Queues
      • Picking a Task
      • Case View
      • Completing a Task
      • Task History
      • Searching Tasks
      • Searching Cases
    • Managing Repositories
      • Moving Files Between Repositories
      • Copying Configurations and Rule Sets Together
      • Copying Dependent Rule Sets
      • Backing up
      • Moving Entire Repositories Between Consoles
      • Setting Repository Imports
      • Specifying Repository Rule Group Restrictions
    • Version Control and Restoring Files
    • Using the Portal UI
      • Using the Start Menu
      • Positioning Windows
      • Resizing Windows
      • Maximize, Restore and Minimize
      • Storing and Switching Desktops
    • Enterprise Level Projects
      • Creating a Project
      • Working with Tasks
      • Closing a Task
      • Closing a Project
      • Project Assistants and Work Output
    • Administration Functions
      • Extensions
      • Custom Function Setup
      • Setting up a Database
      • Input Adaptors
      • Credential Vault
      • Log Adaptors
      • Server Definitions
      • Managing Large Clusters
      • Setting up the Flight Recorder in the Console
      • Creating a Case Manager Definition
      • Work Output
      • Project Definition
      • Managing Users
      • Managing User Roles
      • Managing Access Rules
      • Authenticating via LDAP
      • Authenticating via SAML
      • Audit Log
    • Installation and Configuration
      • Installing on macOS
      • Installing on Red Hat Enterprise Linux
      • Installing on Windows - Quick Start
      • System Requirements
      • Server Installation
      • Understanding the Console Deployment Structure
      • Console Server Type Configuration
      • Console Server Configuration
      • Removing Other Unnecessary Components
      • Installing Inline with an Existing Application
      • Installing a Stand-alone Multi-Protocol Feed Server
      • Installing a Stand-alone Built in Forwarding Proxy
      • Creating a Cluster Slave Console Instance
      • Understanding the magic.properties Configuration Settings
      • Port Numbers and how to Change Them
      • Dealing with Internal Failures
      • Linking Multiple Applications Together into a Single Server
      • Monitoring by External Systems
      • Installation Considerations for Portals
      • Docker Configuration Guide
      • AWS User Deployment Guide
      • Google Cloud User Deployment Guide
    • Creating a Stand-alone Built in Forwarding Proxy
      • Installing with Ubuntu and mySQL
    • Setting up a DNS override for Data Loss Prevention
      • Deciding what to Override
      • Installing a DNS Server for the Overrides
      • Setting the Override
      • SSL Considerations
    • Backup and Restore
      • The Server Component
      • The Console Component
    • Third Party Software
      • Open Source Components
      • Closed Source Licensed Components
      • Licenses
  • PROGRAMMER'S GUIDE
    • Composable Agentic Platform Concepts
    • Setting up a Development Environment
    • Creating a Rule
    • The Kapow Extension - Example
    • Java Documentation
  • CONSOLE WIZARDS
    • Create REST with JSON service
    • Create JDBC database definitions
    • Create data set maintenance custom function
  • BEST PRACTICES
    • Naming Conventions
    • Comments & Descriptions
    • Application Configuration
    • Rules & Rule Sets
    • Repositories
    • Data Manipulation
    • Internationalization
    • Debugging
    • Performance Testing
    • Tag Replacement Techniques
    • Best Practices Repository
  • GUIDES
    • Hello, World!
    • TCL Script Writer Reference
    • Windows Automation Reference
    • Browser Certificate Installation Guide
    • Web Development Guide
    • Using the Push Notification Framework
    • Raspberry Pi with PiFace Reference
    • TomorrowX Portal User Guide
    • Examples
      • CSRF attack prevention
      • Two Factor Authentication
      • Frame Busting
      • Google Analytics
      • DNS Multi Protocol
      • Customer Satisfaction Survey
  • CHANGE HISTORY
    • New in Version 8
    • New in Version 7
    • New in Version 6
    • New in Version 5
    • New In Version 3.1
    • New In Version 3.0
  • 🌏TIME ZONES
  • ❓TROUBLESHOOTING
    • DB2 SQL error: SQLCODE: -443
    • Error 404 when trying to access the console
    • java.io.IOException: Too many open files
    • Accidental deploy to console server
  • ⚙️BROWSE COMPONENTS
  • EULSA
Powered by GitBook
On this page
  • Separation of code and data.
  • Content Files
  • Data Files
  • Data Sets
  • Data Input
  • Memory Considerations
Export as PDF
  1. BEST PRACTICES

Data Manipulation

PreviousRepositoriesNextInternationalization

Last updated 1 year ago

Separation of code and data.

Avoid hard-coded html in the rules, instead use tag replacement techniques or templating techniques (custom templating, template engines, frameworks or CMS’s) that help separate code (business logic) from data (html output).

This helps web designers to focus only on the code relevant to the user interface and not any business logic.

Custom templating should be designed in such a way that it is directly related to the data output

with the use of “if”, “while”, “for”, “foreach”, “switch” control structures

and it should be readable.

The selected templating technique should be consistent throughout a solution, avoiding confusion.

Two tag replacement techniques are detailed in the Appendix A. For more advanced options, like the use of control structures, a templating engine should be used.

Content Files

Include static files, such as js/css libraries in the “Content Files” node.

File and folder organization should be agreed upon by the team, and remain easy to understand and structured.

Data Files

Include data files, such as html, CSV files in the “Data Files” node. A benefit of reading from “Data Files” is that it will be memory cached, and you can update it without redeploying all rule sets, but simply by deploying the data file on its own.

Data Sets

For smaller applications Data Sets are a preferred way to store data, instead of using SQL statements. Data Sets are incredibly flexible and do not require modifications to the schema for every new field or feature.

See the Data Set rules to study an example.

Data Sets are more than adequate for anything up to about 100,000 elements and they offer flexible “key-on-anyfield” and “add a new field any time you like without stopping” capabilities.

Data Input

Custom functions in the Tomorrow Software console can be used as an interface for corporate end-users who have been assigned the role of application/user data maintenance, such as localization data, selection list data, user data, csv files or databases.

User access to desired custom functions can be granted to users through the Tomorrow Software console, by a Tomorrow Software administrator.

Memory Considerations

Consider increasing maximum memory usage when reading from large files that their content needs to be saved to a variable for further processing (i.e., “File Reader” rule), by changing java startup properties in your Tomorrow.bat file. If the file just needs to be forwarded to a http client, then use the “HTTP File Response” rule.

maintain data file example