Skip to content

Administration Guide

Administration Guide

Cyberquest overview

Cyberquest is an innovative data security analytics platform designed to provide comprehensive auditing and security capabilities for small, medium and enterprise networks. Cyberquest has been designed to function as an agile, scalable business platform that intelligently collects and correlates data in an organization's IT infrastructure and works with it to address any type of present or future threat which a business could encounter.

Cyberquest is highly scaleable and can be configured to suit many organization size and use cases and easily integrates with all security solutions on the market, irrespective of their classification. Cyberquest is a true aggregator of security data coming from either Security Information and Event Management software (SIEM), firewalls, intrusion prevention and detection platforms, or email security and endpoint security solutions. In addition, Cyberquest can collect, correlate, and provide useful insights on heterogeneous data generated by network equipment, servers, databases, and applications, which makes it an invaluable operational management tool for your security and administrative teams.

Core Capabilities

  • Collect: gather all security and relevant data sources from your IT infrastructure;

  • Correlate: add threat intelligence security data for offline or online correlation;

  • Detect: quickly identify the most significant threats to your network;

  • Visualize: monitor accurately within a single point of access and get specific alerts;

  • Respond: Security Orchestration, Automation, and Response (SOAR) capabilities are embedded in the solution;

  • Vulnerability assessment: with OpenVAS integration.

Cyberquest aggregates and monitors all activity in your infrastructure and, with real-time alerts, SOAR, and vulnerability assessment capabilities, delivers detailed information and fires response for vital changes and activities - as they occur. Instantly know who, what, when, where, why made a change, and then turn that information into intelligent, in-depth forensics, enhanced with additional data from the entire environment, make that information available for auditors and security officers and generate automated actions as response to the risks associated with day-to-day modifications.

Concept and availability

Cyberquest seamlessly integrates into your existing IT infrastructure and delivers real-time user behavior and data monitoring, threat detection, data analytics and correlations, security information and event management, in a single platform, enabling you to:

  • Have unified and increased infrastructure visibility for security management

  • Ensure and track regulatory compliance, security audits and policy

  • Reduce organization threat surface quickly and accurately

  • Optimize and generate predefined, ready-to-go reports

  • Improve your existing security and event management solution's response capabilities to incidents

Cyberquest is an appliance type of product (hardware & software) supporting multi-redundant topologies and that can be scaled horizontally by installing any number of processing nodes, or vertically by adding processing resources. Given its deployment flexibility, the solution can be easily architected to meet multi-site deployment challenges. The solution is also available through Software-as-a-Service offering.

Its main functionalities, given by multiple modules, are:

  • Normalizing available information from different systems in its own format through special dedicated connectors;

  • Dashboards module: provides a visual representation of events, aggregated by different criteria;

  • Browser module: provides access to all events and viewing the details

  • Reporting module: provides reporting functions;

  • Alerting module: provides real time alerting for configurable situations with configurable response actions (SOAR);

  • Investigation module: a visual way of seeking events and investigating situations;

  • Case management module: a case management function for collaborative investigation activities;

  • Administrative module: ensures configuration and management functions for the application;

  • Data Transformation System module: a proprietary module that is responsible with different functionalities of Cyberquest such as event enrichment, data anonymization etc.

  • Vulnerability Assessment Module: provided by integration of OpenVAS (https://www.openvas.org/)

High-level data flow description

Nextgen Cyberquest is a dedicated security analytics platform intended to be used by IT security officers. Therefore, Cyberquest helps companies to be more secure and also compliant to internal and industry regulations by doing collection of high volumes of disparate data from infrastructure and third-party security solutions, aggregating and enhancing collected data, and presenting security personnel with useful information on possible threats and risks -- all in real-time.

Initially, real-time data is collected from various sources using Cyberquest's WMI, syslog, NetFlow, ODBC or file-level gathering capabilities. Data is organized in queues sent to a Data Acquisition Service (DAS), which applies acquisition rules and then sends raw data to a Data Transformation Service (DTS). DTS is responsible for parsing data and generate real-time alerts.

Once parsing rules are applied, transformed data is applied with retention rules. Retention rules will tell if data is stored in the Online Storage or Archive Data Storage. The major difference between the two types of storage is access speed. Online Storage applies indexing on uncompressed data, which makes any information available in term of seconds, with the cost of space. Archive DataStorage is designed for long term retention of data, without imposing a limit to the maximum volume of that data. The archive stores data in compressed and encrypted files; a compression ratio of 1:20 is the norm.

Alt Image

Online and Archive storages exchange data depending on need. When a certain information is requested, data is automatically extracted and imported into Elastic Search nodes for processing. Correlation is performed by a Cyberquest Server and resulted information is presented in dashboards and reports.

Cyberquest Web Interface is the central module used for both management and utilization of platform. Web Interface uses a web frontend allowing administrators and operators to interact with Nextgen Cyberquest. Depending on the access level allowed, a user will be able to access Reports, Dashboards, Investigations, Browser and Alerts modules and take benefit of the entire set of security analytics.

Services and Components

Data Collection

Cyberquest receives data from a wide range of devices. The data collected is of various formats, but mainly separated into sections as it follows: collection of event data, flow data, vulnerability assesment information (VA), and other data types relevant for security analytics.

Event data collection: events are being collected from data sources that log security-based events, such as: routers, servers, firewalls, intrusion detection systems (IDS) or intrusion prevention systems (IPS), network switches, active directory and other networking equipments.

Flow data collection: network traffic information is collected from IPFIX and Netflow network protocol (versions 5, 9, and 10).

Vulnerability assessment (VA): relevant security states.

Data Collection is a distributed service, engaged with several components that can reside within the main appliance, or independently to accommodate star or cloud architectures. Once collected, data is encrypted and compressed, then transported by the message queuing sevice to the data aquisition service.

If data cannot be delivered to message queuing service, one of the following agents will perform local caching for later delivery:

  • Windows Agent, capable of storing up to 1 million events.

  • Data Server performs caching behaviour when needed, by creating files on the hosting station in order to capture all events, without any event being lost during the process.

Windows Agent

Windows Agent is a data collection component, interacting directly with event sources. It can be installed on workstations and servers. The minimum operating system version is Windows 7 (on workstations) and Windows Server 2008 (on servers), and has a listed software dependency on Microsoft .NET Framework 4.5.2.

The agent uses configuration templates that can be associated to one or multiple data sources. The templates contain settings specific to data sources: credentials, field mapping, event filtering, data processing scripts, database queries and more.

The agent addresses the following data sources by default:

  • Local and remote WMI (Windows Management Instrumentation) collection:

  • Windows Security logs

  • Windows Application logs

  • Windows System logs

  • Other application and services logs in Windows standard format

  • Local and remote MS SQL collection:

  • Queries for incremental data collection from database tables containing application logs (minimum version supported is MSSQL Server 2005)

  • SQL Server audit (minimum version supported is MS SQL Server 2008) by default

  • Local and/or remote Oracle collection:

  • Queries for incremental data collection from database tables containing application logs (minimum version supported is Oracle 9i)

  • Native Oracle instance audit; by default, the solution provides templates for Oracle 9i, 10g, 11g and 12c

  • Custom data collection (which requires parameters and mappings to be configured in each customized template):

  • Local and/or remote collection for ODBC (open database connectivity) sources supporting 64-bit ODBC drivers

  • Local and/or remote collection for MySQL, MariaDB, PostgresSQL, MS-SQL sources, including queries for incremental data collection from database tables containing application logs and native platform audit

  • Local collection of timestamped files in CSV/TSV format

  • Local collection of parsable custom files of any file type, for which the agent uses a pre-processing script

  • HTTPS API-based data

  • Others

Data Server

Data server is a distributed service engaged with several components that can reside within the main appliance, or independently to accommodate star or cloud architectures. The service is used to perform data pre-parsing on syslog native format compatible with RFC 5424 and RFC 3164 standards. The system can also receive and process NetFlow data (NetFlow versions 5, 9, 10 and ipfix) and perform NetFlow stiching called Biflow.

Also, Data Server acts as a passive agent for syslog and NetFlow messages received by Cyberquest. By default, the service is configured to listen on UDP/TCP 5140 port for syslog events and UDP 2055 for NetFlow packets.

Services for data transport and processing

Message Queueing (MQ) Service

MQ Service is an intermediary for messaging. It gives a common platform to the Cyberquest components to send and receive messages that are safe to live until received. MQ Service transports messages from data collection agents to processing and storage services. This service is based on advanced message queueing protocol via RabbitMQ software.

Cyberquest uses the following queues for interprocess communication:

  • events: used to collect data from agents

  • DataStorage: used to process data to Data Storage Service

  • heartBeat: used to process data that is relevant to data source status

  • DataCorrelation: used to store and process data to Data Correlation Service

  • jobCommands, used to communicate between the web interface and DataStorage service to start, stop, and view the status of the current jobs.

All data within MQ Service is encrypted and archived.

MQ Service is automatically installed in all-in-one deployments and uses 3 GB of memory. For increased efficiency, in large environments we recommend deploying the service on a standalone system. Also, for high availability and load balancing scenarios, the queues can be replicated to a secondary MQ Service.

Data Acquisition Service

This service processes the data received on the events queue from MQ Service. Once data is decrypted and decompressed, it is passed on to a series of modules that add or extract additional data in the following order:

  • Asset Manager Module: the service passes the event to the module of automatic detection and corellation of assets, and enriches the event with the information stored in the Agent Management Cyberquest section (such as geolocation, asset security score, time skew, owner identification, and so on).
  • Mapping Manager Module: helps expose forensic information in events so that users are able to pin-point the 5 w-s (who, what, where, when, why).
  • Default Data Parsing Module: built-in tag based data parsers, with proceeses like extraction and convertion of information, event cross-correlation, etc.
  • Geolocation Appending Module: using GeoIP2 database, it searches SourceIP and DestinationIP addreses, appending the country name, country code, city, and geocation points to them.
  • Data Transformation Service (DTS) Module: based on complex filters, the event can match a data acquision rule (DA Rule), which can apply a data transformation script enabling the user to customize the event even further (interacting directly with the event), and/or select the long-term storage. The user can also send alerts, emails, temporarily store/retrieve information (to cross-correlate with future events), or search for historical events in the short-term storage (ElasticSearch), or event dropping.
  • Data Output Module: the processed event is prepared to be sent to ElasticSearch via HTTP/HTTPS, and it is also encapsuled and encrypted/archived to be sent to our message queueing service (RabbitMQ).

Data Acquisition Service can be installed on multiple machines, allowing for load balancing, high availability and vertical expansion (work load division).

Data Correlation Service

This service allows the user to correlate events based on data correlation rules that accomodate different needs, such as: brute force attack prevention, abnormal user behaviour, virus attacks (and/or propagation), application missbehaviour, and so on.

The user can configure multiple types of alerts: - Single Event (rule) Alert, Such as administrator logons during non business hours. - Correlated Events: multiple rules tied together by different fields, which are performed in sequence during a speciffied time frame. - Summary Alerts.

For Correlated Events option, there can be different methods of Alerts and/or Actions that the user can trigger: - Single event trigger, which will trigger on first event (that matches) correlated with previous event or chain of events. - Multiple events trigger which will trigger if multiple events match the correlation rules with regards to the following settings: Rule Trigger type, Min. Treshold, Max Treshold, TTL (time to live in seconds) and Pivot Field. The Rule trigger typs are: - Trigger based on count until Max treshold is achieved before TTL expires OR the TTL expires and Min treshold is reached. - Trigger based on Sum of a numeric value offered by Pivot field setting, if it is greater or equal to Max Treshold value before TTL expires OR is greater or equal to Min Treshold value when TTL expires. - Trigger based on Average value of a numeric value offered by Pivot field setting if it is greater or equal to Min Treshold value when TTL expires.

Data Correlation comparison rules are: - =: Matches if the fields are strinctly equals; - : Matches if the fields are strinctly not equals; - isInList: Matches if the value is contained in a previously defined list. Additionaly, you can add terms to be verified (or only terms can used, instead of the pre-defined lists). To indicate the beginning of a new term, use a new line (press ENTER). To indicate a list name, put @ before inserting the list name. - isNotInlist: Matches if the value is not contained in a previously defined list. Additionaly, you can add terms to be verified (or only terms can be used, instead of the pre-defined lists). To indicate the beggining of a new term, use a new line (press ENTER). To indicate a list name, put @ before inserting the list name. - StartsWith: Checks if the field starts with a specific value. - EndWith: Checks if the field ends with a specific value. - Contains: Checks if the field contains a specific value. - <: Checks if the first value is less than the second value. The compared values have to be of the same type (strings, integers, IPv4, or of time/date ANSI format). - : Checks if the first value is less or equal to the second value. The compared values have to be of the same type (strings, integers, IPv4, or of time/date ANSI format). - > Checks if the first value is greater than the second value. The compared values have to be of the same type (strings, integers, IPv4, or of time/date ANSI format). - Checks if the first value is greater or equal to the second value. The compared values have to be of the same type (strings, integers, IPv4, or of time/date ANSI format). - range: Checks if the first value is between a range of two values. The compared values have to be of the same type (strings, integers, IPv4, or of time/date ANSI format).

Data Correlation Rules are defined as: - Field conditions that compare a field value with a chosen value; - Report conditions that compare the event with a predefined report the user can choose from; - Correlation condition, where the user can compare a value from current event with a value from the events triggered in the chain of events.

Composition rules

The user can select and combine the main logic operators: AND, OR, NOT. The NOT logic operator exists on each condition field, negating the outcome of that condition. AND and OR are used to compose the current rule. An associative property is defined when switching between AND and OR logic operators button, grouping everything before and after that operator.

Alt Image

Cyberquest events have the following fields:

Event Type String EventType int Description
Error 1 An error event. This indicates a significant problem the user should know about; usually a loss of functionality or data.
FailureAudit 16 A failure audit event. This indicates a security event that occurs when an audited access attempt fails; for example, a failed attempt to open a file.
Information 4 An information event. This indicates a significant, successful operation.
SuccessAudit 8 A success audit event. This indicates a security event that occurs when an audited access attempt is successful; for example, logging on successfully.
Warning 2 A warning event. This indicates a problem that is not immediately significant, but that may signify conditions that could cause future problems.

Event objects are called by choosing the event name (out of the list below), followed by dot (.), and field type. Objects can be of the following:

_network
_geoLocation
_attack
_agent
_incident
_asset
_malware
_dataSource
_Timestamp
_forensics
_event

Therefore, for each object, the following events can be called:

Event Type Description
CapturedImage binary Stores encrypted picture file formats
Category string Stores the event category type
Computer string Stores the name of the computer on which the event was triggered on
Description string Stores a decription of the event
DestIP string Stores the IP address of the device to which the packet is being sent
DestIP_Country_Code string Stores the country code of DestIP
DestIP_Country_Name string Stores the country name of DestIP
DestMAC string Stores the destination MAC address
EventID long Stores the unique ID of the event
EventLog string Stores the event log
EventPath string Stores the event path
EventType long Stores the event type
GMT date Stores the date in DateTime ANSI format
ID string Stores the ID
IP string Stores the IP
IsIncident boolean Stores information as to if the event is categorized as a security incident in CONST_EL_TYPE_BOOLEAN
LocalTime date Stores the local time in DateTime ANSI format
N1 ... N40 long Stores general purpose numeric fields
PlatformID string Stores the platform id
PostDtsSHA256 string Stores the log hash after passing through Data Transformation Service
PreDtsSHA256 string Stores the log hash before passing through Data Transformation Service
RawData string Stores raw data
ReceivedTime date Stores the received time in DateTime ANSI format
S1 ... S150 string Stores general purpose string fields
SecondaryTag string Stores a secondary tag
SessionID string Stores the session ID
Source string Stores the source
SrcIP string Stores the IP (Internet Protocol) address of the device sending the IP packet (the IP unit of data transfer)
SrcIP_Country_Code string Stores the country code of SrcIP
SrcIP_Country_Name string Stores the country name of SrcIP
SrcMAC string Stores the source MAC address
Tag string Stores a tag
Tenant string Stores the tenant
TimeOfDay long Stores the time of day
UserDomain string Stores the user domain
UserName string Stores the username
VersionMajor long Stores the version major
VersionMinor long Stores the version minor
content string Stores content
_Timestamp.SkewedOffset long Stores the difference between real time and machine time
_Timestamp.Time long Stores a scalar real number, which represents the number of seconds that have passed since 00:00:00 UTC Thursday, 1 January 1970
_Timestamp.TimeZoneOffSet long Adds 80 seconds to the GMT
_Timestamp.isDST boolean Stores the the summer time if applied or not
_agent.GUID string Stores the agent globally unique identifier
_agent.Name string Stores the name of that agent concerned
_agent.Site string Stores the location of the agent concerned
_asset.Application string Stores the application name
_asset.Criticality long Stores the security level (as rating)
_asset.GeoLat GEO Stores the decoded latitude from IP address
_asset.Name string Stores the actual name of asset
_asset.Owner string Stores the name of the owner
_asset.Project string Stores the name of the project
_asset.SecurityValue long Stores the security level
_asset.Site string Stores the location where a certain action has happened (city)
_asset.URGeoLongL GEO Stores the decoded longitude from the IP address
_attack.DestIP string Stores the IP address of the device to which the packet is being sent
_attack.GeoCity string Stores the decoded City from IP address
_attack.GeoCountry string Stores the decoded Country from IP address
_attack.Host string This event is triggered if a computer, or other device communicates with other hosts on a network (includes clients and servers), that send or receive data, services or applications
_attack.GeoLat GEO Stores the decoded latitude from IP address
_attack.GeoLong GEO Stores the decoded longitude from IP address
_attack.Method string Is a particular procedure for accomplishing or approaching something, especially a systematic or established one.
_attack.Object string Network objects are used to categorize IP addresses into different types of network entities
_attack.OtherInfo String Stores extra information about the network
_attack.Result boolean Stores the result of the attack
_attack.SrcIP string Stores the source IP of the attack
_attack.TriggeredRule string Defines the conditions under which a trigger action is to be executed.
_dataSource.Name string Stores the name of the data source
_dataSource.SecurityAppliance string Stores the physical name of the data source
_dataSource.Version string Stores the data source version
_event.Category string Stores a category, which is assigned by Cyberquest for each event
_event.Result boolean Stores the result of the event (success or failure)
_event.SourceObject string Stores information as to where does the object come from; its origin
_event.SourceUser string Stores information as to where does the user come from; its origin
_event.SubCategory string Stores a subcategory, which is assigned by Cyberquest for each event, depending on the main category
_event.TargetObject string Stores the destination of the object
_event.TargetUser string Stores the destination of the user
_event.URL string Stores the Uniform Resource Locator (URL) of the event
_forensics.What string Stores the description of the action
_forensics.When string
_forensics.Where string Stores the location where the event occurred
_forensics.Who string Stores information as to who created the event
_forensics.Why string Stores information as to why was the event created
_geoLocation.DestIPGeoCountry string Stores the destination IP coordinates of the country
_geoLocation.DestIPGeoPoint geo_point Stores the destination IP coordinates of the point
_geoLocation.DestIPGeocity string Stores the destination IP coordinates of the city
_geoLocation.Host string host
_geoLocation.SrcIPGeoCountry string source Stores the IP coordinates of the country
_geoLocation.SrcIPGeoPoint geo_point Stores the source IP coordinates of the point
_geoLocation.SrcIPGeocity string Stores the source IP coordinates of the city
_incident.Category string Stores a category, which is assigned by Cyberquest for each incident
_incident.Impact string Measures the extent of the incident and the potential damage caused by the incident, before it can be resolved
_incident.Score long Stores the incident score
_incident.SubCategory string Stores a subcategory, which is assigned by Cyberquest for each incident, depending on the main category
_malware.DeliveryMethod string Stores the delivery method through which the malware was sent (e-mail, file, etc.)
_malware.Name string Stores the name of the malware
_network.AplicationName string Stores the name of the application
_network.DestIPv4 ip Stores the destination IP (IPv4)
_network.DestIPv6 string Stores the destination IP (IPv6)
_network.DestInterface string Stores the destination interface
_network.DestPort long Stores the destination port
_network.FlowID string NetflowID
_network.PostNATDestIPv4 ip Stores the destination IP(IPv4) after network translation
_network.PostNATDestIPv6 string Stores the destination IP(IPv6) after network translation
_network.PostNATDestPort long Stores the destination port after network translation
_network.PostNATSrcIPv4 ip Stores the source IP(IPv4) after network translation
_network.PostNATSrcIPv6 string Stores the source IP(IPv6) after network translation
_network.PostNATSrcPort long Stores the source port after network translation
_network.Protocol string Stores the protocol
_network.ReceivedBytes long Stores the received bytes
_network.SrcIPv4 ip Stores the destination IP(IPv4)
_network.SrcIPv6 string Stores the source IP(IPv6)
_network.SrcInterface string Stores the source interface
_network.SrcPort long Stores the source port
_network.TransferedBytes long Stores the transferred bytes

Data Correlation Service can be installed on multiple machines, allowing for load balancing, high availability and vertical expansion (work load division).

Administration Service

Administration Service checks data collection status and raises alerts when data no longer reaches processing server. Also, it checks services status and alerts when operational and availability issues occur.

Storage services

Data Storage Service

Data Storage Service performs long-term storage of received data and events.

Data Storage Service follows general deployment architecture to achieve high availability and load balancing requirements. Data synchronization in distributes scenarios is done using RSync technology between nodes, so that no information is ever lost.

Data Storage Service recives the logs from the Data Aquision Service, and processes to archieve them: logs are encripted, digitally signed, and then compressed at a ratio of 1:20.

Data Storage Service also ensures importation of all data into the Cyberquest online database. Importing of this data is triggered only when creating an import job manually. To do that, go to Settings > Jobs > New Job > select Import Job option.

Other services and components

Cyberquest uses additional services, such as: - NoSQL Service, which performs short-term storage of received data and events. This service is based on ElasticSearch technology and uses ElasticSearch deployment and distribution architectures. - MariaDB, which provides database layer support for Cyberquest. The database is used only for storing configurations, no data being stored on the database; - NginX, which provides web application layer support for the web interface. It uses HTTPS protocol to present the web interface to users; - PHP-FPM, which is the PHP interpreter for the web server. - Redis, which is an in-memory data structure store used for storing lists; - Other linux native components (cron, iptables, and more)

Cyberquest Web Interface

Accessing the web interface

Web Interface is a consolidated web frontend hosting all administration and operation functionalities of Cyberquest. The web interface is compatible with all major browsers on the market.

To access Web Interface, open a web browser and type the application's address or DNS name. The default address initially assigned to Web Interface is https://CyberquestIPAddress.

The browser automatically redirects you to Cyberquest's authentication page:

Alt Image

User authentication

Authentication can be accomplished in one of two ways:

  • Using a local user defined in the application; <!--
  • Using a company's Active Directory user. This facility allows authentication with Active Directory credentials when LDAP integration has been configured within application. The user must belong to one of two Active Directory security groups: "Cyberquest Administrators" or "Cyberquest Users".--> After typing your username and password, press Alt Image button.

The initial authentication is performed under the default administrative account. When authenticating as administrator, an additional confirmation box is displayed. This additional authentication step was introduced to notify on indiscriminatory access to the entire platform configuration and to require user confirmation of acknowledgement. Superadmin activity should be performed with maximum responsibility and knowledge of platform's administration. Wrongfully changing configuration, rules and retention policies can break access to analytics data, delete or damage objects, and more important, can cause permanently loss of history data.

Alt Image Alt Image

If you agree authenticating as an administrator, press Alt Image button and Web Interface will open. If you want to go back to login and authenticate as an operator, press Alt Image button.

Web Interface Overview

Once authenticated, Nextgen Cyberquest Web Interface will open. By default, Dashboards module is displayed. Depending on each user's access permissions, the interface may differ. Below we are describing user experience and interface functionalities when authenticating as an administrator.

The Web Interface can be split in several areas:

Alt Image

Module Area

Alt Image

From top-left section of the Web Interface you can select the application module to be displayed in main operation area:

  • Dashboards is the default module that loads when first authenticating to application. It allows an operator to quickly view information contained in the online repository, and action on contained graphical objects

  • Reports is the reporting module proprietary to the application. It contains all predefined and custom reports for general use and also, reports defined for the authenticated operator

  • Investigations module (or mode) is intended to represent graphically the audit information from the application. This mode allows native correlation of data and connecting apparent relational events. This serves to create bonds between diverse events and fields/strings.

  • Browser module (or mode) is intended to display the log information present in the system.

  • Alerts module (or Alerting mode) manages alerts and alert correlations, and allows users to start full investigation processes from an initial point -- the base alert displayed in Main Operation Area. Alerts button

By clicking on Alt Image logo displayed in the top-left corner of the Web Interface, you will be taken to the "home" screen that is displayed after logging in to application.

Main Operation Area

Main Operation Area is the place where people accessing the application can perform their activities. This area is specific to each accessed module (or mode) and options being available depend on user's assigned permissions. Depending on each module capabilities, Main Operation Area may contain per user personalized content -- like custom dashboards and reports.

Available content and options are detailed within each module chapter in Cyberquest 2.15 User's Guide.

Performance Area

Performance Area in the top-right side of the Web Interface maintains three indicators updated in real time:

Alt Image CPU -- displays Cyberquest Web Application Server current CPU load. Pointing on the color-filled part of the graph opens a tooltip with the actual value of current load
Alt Image Memory -- displays Cyberquest Web Application Server current memory load. Pointing on the color-filled part of the graph opens a tooltip with the actual value of current load
Alt Image Disk -- displays Cyberquest Web Application Server current disk load. Pointing on the color-filled part of the graph opens a tooltip with the actual value of current load

User Enabler Area

User Enabler Area in the top-right side of the Web Interface comprises three action buttons as follows:

  • Stats Alt Image button opens a quick pop-up with statistical information on processed data. The following information is provided:

    Alt Image

  • Total events -- total number of events currently stored in the online repository

  • Last hour events -- total number of events collected in the last hour

  • Last day events -- total number of events collected in the last day

  • Total alerts -- total number of alerts currently managed by the Application Server

  • Last hour alerts -- total number of alerts raised in the last hour

  • Last day alerts -- total number of alerts raised in the last day

  • User Alt Image button opens User drop-down menu containing the options described below:

Alt Image

  • Change password option opens Change your password window, where currently logged in user can change his password.

  • Executed schedules option opens My Executed Schedules report listing all schedules executed by the currently logged in user

  • Case Management option opens Case Management module for the currently logged in user

  • Logout option logs out currently logged in user

  • Settings Alt Image button opens Settings drop-down menu containing the options described below:

Alt Image

  • Users and Groups > Users and Users and Groups > Groups are options allowing an administrator to view, add, edit or delete users and groups. Additional actions are available for users: change password, activate or inactivate, copy dashgroups to users.

    Each of the Application Settings option opens Application Settings configuration page allowing an administrator to configure in detail the main Cyberquest settings. The page presents configuration capabilities for:

    • Active Directory
    • Administration
    • Agents
    • Integrations
    • Teams
    • Jira
    • Slack
    • Alert templates
    • Assets
    • Asset Groups
    • Asset Groups Types
    • Customize
    • Data Acquisition
    • Data Correlation
    • Data Server
    • Data Storage
    • ElasticSearch
    • Email
    • Reports Export
    • Retention Period
    • Tenants

  • Event dictionary option opens Event Definitions configuration page allowing an administrator to list all event definitions, add a new event definition or import a definition from an external file, or perform actions on existing event definitions. Possible actions are export, edit and delete.

    Alt Image

  • Management > Dashboards option opens Dashboards configuration page allowing an administrator to list all defined dashboards, import a definition from an external file, or perform actions on existing dashboards. Possible actions are edit and delete.

  • Management > Filters option opens Filters configuration page allowing an administrator to list all defined filters, add a new filter, or perform actions on existing ones. Possible actions are edit and delete.

  • Management > Objects option opens Object Management configuration page allowing an administrator to list objects or add a new object. Possible actions on listed objects are edit and delete.

  • Management > AgentManager option opens the Agent Manager configuration page allowing an administrator to register a new Windows agent. Possible actions are edit, deploy and delete.

  • Management > DataSourceManager option opens the Data Source Manager configuration page allowing an administrator to add a data source.

  • Management > CredentialManager option opens the Configured Credentials page allowing an administrator to add a new credential. Possible actions are edit and delete.

  • Management > CQVulnerabilityManager option opens the Vulnerability Manager page allowing an administrator to update the list of vulnerabilities.

Alt Image

  • Alerts > Summary option opens the list of custom summary alerts in Alerts module, allowing an administrator to list alert templates, create a new alert template or create a new registered summary alert. Possible actions on listed summary alerts are activate/inactivate, edit and delete.

  • Alerts > Notification templates option opens Alert Templates configuration page, allowing an administrator to create a new alert template or action on listed alert templates. Possible actions are edit and delete.

  • Alerts > Realtime option opens the list of defined alerts in Alerts module, allowing an administrator to create a new alert definition or import alert from external file, and to perform actions on existing alert definitions. Possible actions are edit, export and delete.

    Alt Image

  • Rules > Filter Rules option opens Filter Rules configuration page allowing an administrator to create a new filter rule, import a filter rule from an external file or perform actions on existing ones. Possible actions are activate/inactivate, export, edit and delete.

  • Rules > DTS Objects option opens DTS objects configuration page allowing an administrator to create and import a DTS object from an external file or perform actions on existing ones. Possible actions are activate/inactivate, export, edit and delete.

  • Rules > DA Rules option opens DA Rules configuration page allowing an administrator to create and import a data acquisition rule from an external file or perform actions on existing ones. Possible actions are activate/inactivate, export, edit and delete.

    Alt Image

  • Jobs > Jobs option opens Jobs configuration page allowing an administrator to create a new job or perform actions on existing ones. Possible actions are activate/inactivate, run, edit and delete.

  • Jobs > Jobs Executions option opens the list of job executions. You can delete a job execution and see the execution status for each listed job.

    Alt Image

  • Network Applications option opens Network Applications configuration page allowing an administrator to create a new network application, search in list or perform actions on existing ones. Possible actions are edit and delete.

  • Data Storages option opens Data Storages configuration page allowing an administrator to create a new data storage or perform actions on existing ones. Possible actions are activate/inactivate, edit and delete.

  • Data source status option opens a report of all data sources and their status. The report allows for data sources to be deleted and alert time to be changed. Each data source is presented with a status. The page includes a search field and possibility to sort by any column. The report can be customized in terms of details included or excluded, and exported in CSV format.

Alt Image

  • Batch Fields Checker option opens Batch Fields Checker window allowing you to upload a text file and execute batch checking. Result can be exported in CSV format.

Changing User Password

Once authenticated, a user can change his password from User menu. This is a strongly recommended action after the first login, and it can be performed at any time forward.

In order to change your password, access User > Change password option. Change your password window opens:

a. In Old Password field, type your current password

b. In New Password field, type the new password. Make sure you follow the complexity requirements set for the specific company environment

c. Repeat the new password in Confirm Password field

d. Press Alt Image button to save the new password and close the window or press Alt Image button to close the window without saving changes. As an option, you can close the window without saving changes by clicking the Alt Image mark in top-right corner.

Alt Image

e. After changing password, it is recommended to perform logout by clicking User > Logout in User menu.

An administrator with user management privileges can change his password and can also change passwords for any other user. In order to do that:

a. In Settings menu, click Users and groups > Users. Users configuration page opens.

b. Click on Alt Image button for yourself or the user for which you need to change the password. A different Change user password opens

c. Type in the new password in Password and Password Confirm fields.

d. Press Alt Image button to save the new password and close the window or press Alt Image button to close the window without saving changes. As an option, you can close the window without saving changes by clicking the Alt Image mark in top-right corner.

Alt Image

e. Instruct the user to logout from application and then log back in.

Managing Users and Groups

Role Based Access Control (RBAC)

User accounts created can be configured to access components based on the user role assigned to their account. You can add or edit user roles and user accounts as needed.

Add or edit User Roles

User roles are assigned to user accounts to control access in Web Interface. You can add or edit user roles as needed. Roles are assigned at group level.

In order to add or edit user roles:

a. Login to application with an administrative account.

b. Navigate to Settings by expanding Alt Image in the top-right corner of the interface, then click Users and groups > Groups.

Alt Image

c. In Groups window, click the option New Group.

Alt Image

d. Add group window opens. In Add group window:

In the Name field, provide a name, such as Users Restricted Permissions.

In the Users field select the users that will be impacted by the predefined rules.

In the Assigned Permissions field select the appropriate permissions for the selected users.

In the Data Permissions field select the appropriate data the selected users can view.

Alt Image

e. By default the new group is enabled. Deactivate the group by selecting Alt Image option.

f. Press Alt Image button to add new defined group and close the window or press Alt Image button to close the window without saving changes. As an option, you can close the window without saving changes by clicking the Alt Image mark in top-right corner.

Delete User Groups

To delete a group, navigate to Settings by expanding Alt Image in the top-right corner of the interface, then click Users and groups > Groups. Press Alt Image icon and confirm deletion of desired group. The procedure is irreversible.

Built-in groups cannot be deleted.

Edit User Groups

To edit a group, navigate to Settings by expanding Alt Image in the top-right corner of the interface, then click Users and groups > Groups. Press Alt Image icon. Edit group window is displayed where you can change group name, group members, assigned permissions and data permissions.

Changing group members, assigned permissions and data permissions is done by selecting or de-selecting objects in each drop-down list.

Alt Image

You can also change group status being Enabled or Disabled. Group status can be quickly changed from main Groups window by actioning on Active option button and selecting On or Off. In this case, changes are saved automatically.

Alt Image

For built-in groups you will only be allowed to add or remove members.

f. Press Alt Image button to save changes and close the window or press Alt Image button to close the window without saving changes. As an option, you can close the window without saving changes by clicking the Alt Image mark in top-right corner.

Dashboards migration

Each user can create his own dashgroups containing its own dashboards, but only an administrator can migrate a dashboard from a user to another. After creating a new user, an administrator can copy dashgroups from another user that already has dashgroups configured. To do this, follow the next steps:

a. Login to application with an administrative account.

b. Navigate to Settings by expanding Alt Image in the top-right corner of the interface, then click Users and groups > Users.

Alt Image

c. In Users window, select Copy dashgroups to user

Alt Image

d. Copy dashgroups to user window will open. Check source and destination users from User where dashgroups are copied from and Users where to copy dashgroups drop-down lists. Select desired dashgroups from Dashgroups that are copied drop-down list and press Save to save changes.

Alt Image

e. Logout from the administrative account and login with the new user account. After login is successful, the Dashboards module will show the new dashgroups selected during previous step.

Data permissions

Cyberquest provides data permissions options, which combined with the role-based access features offers a granulized control over the data made available for user members of a group. Data permissions are set at group level.

In order to change data permissions for an existing group, follow these steps:

a. Login to application with an administrative account.

b. Navigate to Settings by expanding Alt Image in the top-right corner of the interface, then click Users and groups > Groups.

Alt Image

c. In Groups window, click Alt Image button for the group containing the user for which you want to change data permissions. Edit group window opens.

d. In Data Permissions field select or deselect on the appropriate data permissions. If no filter is selected, the user will have unrestricted access to all data available.

Alt text

f. Press Alt Image button to save changes and close the window or press Alt Image button to close the window without saving changes. As an option, you can close the window without saving changes by clicking the Alt Image mark in top-right corner.

Configure LDAP authentication

Cyberquest can be integrated with Active Directory or LDAP systems, allowing users to directly authenticate to Cyberquest. To do this, please follow these steps: In Web Interface select Settings > Application Settings > Active Directory. Active Directory configuration page opens.

The following fields can be edited:

  • Active Directory Basedn: Location of the user used to connect to Active Directory. Example: "CN=Users,DC=domain,DC=com".

  • Active Directory Group: The Active Directory group intended for synchronizing users with Cyberquest.

  • Active Directory Password: Account password for the previously mentioned username.

  • Active Directory Port: The port for connecting with Active Directory LDAP service. Default port is 389

  • Active Directory Server (address): The network IP address of Active Directory domain controller to query

  • Active Directory Suffix: FQDN of the Active Directory domain. Example: "domain.com"

  • Active Directory User: The administrative user used to connect to Active Directory. Example: "domain\Administrator". It is also the default user under which Cyberquest will perform event collection from Active Directory infrastructure.

    Alt Image

For each setting, press Alt Image button to save changes and close the window or press Alt Image button to close the window without saving changes. As an option, you can close the window without saving changes by clicking the Alt Image mark in top-right corner.

After authentification, every new user has to be added by a Cyberquest administrator to their first group. After being added to one or more groups, new users can also be promoted as administrators, if needed.

Cyberquest Licensing and Versioning

Cyberquest is available both as free edition or commercial edition. When purchasing the commercial edition, one of the following licences will be generated: Light, Standard, Advanced, Enterprise or Ultimate licence. The table below explains the features and benefits of each:

Free Logger Light Advanced Enterprise Ultimate
Users limit 1 3 5 10 20 Unlimited*
Included cores 4 4 4 6 8 1**
Included EPS 250 250 500 750 1000 Unlimited*
Daily GB of data 1 2 10 160 600 Unlimited*
CPU cores/instance 4 Unlimited Unlimited Unlimited Unlimited Unlimited
Dashboards yes yes yes yes yes yes
Custom dashboards no no yes yes yes yes
Dashboards per dashgroup 6 9 12 15 24 Unlimited*
Dashgroups per instance 2 4 6 8 16 Unlimited*
Data permissions-no. of filters 2 10 Unlimited* Unlimited*
Short term Data Storage (days) 15 30 30 90 Unlimited* Unlimited*
Long term Data Storage (days) 9 180 Unlimited* Unlimited*
Max. no. of Elastic Search Nodes/instance 1 1 2 3 Unlimited* Unlimited*
Max. no. of DTS Objects 4 6 10 15 Unlimited* Unlimited*
Geolocation 🗸 🗸 🗸 🗸 🗸 🗸
Reports 🗸 🗸 🗸 🗸 🗸 🗸
Compliance Reports 🗸 🗸 🗸 🗸 🗸 🗸
Asset Management 🗸 🗸 🗸
Windows Agent Collection 🗸 🗸 🗸 🗸 🗸 🗸
ODBC Data Collection 🗸 🗸 🗸 🗸 🗸 🗸
Syslog Data Collection 🗸 🗸 🗸 🗸 🗸 🗸
NetFlow Data Collection 🗸 🗸 🗸
NetFlow Stitching 🗸 🗸
Multiple Data server for advanced technologies 🗸 🗸 🗸
MultiTenancy 🗸
Simple Alerts (1 step rules) 5 50 Unlimited* Unlimited*
Correlated Alerts (multi step rules) 20 Unlimited* Unlimited*
Summary Alerts 20 Unlimited* Unlimited*
Add-on 2, 4 or 8 Cores-extra fee 🗸 🗸 🗸 🗸 🗸
Add-on 1GB, 5GB, 10GB or 100 GB/day-extra fee 🗸 🗸 🗸 🗸 n/a

*No limit set in the licence. Performance is dependent on the resource allocation.

**The licence is for 1 CPU Minimum licence used is 4 CPU.

Depending on the licensing model you choose, there are also restrictions on the available connectors, nodes and various configuration objects.

To access Licensing module, go to Settings > About. About Cyberquest window opens containing information about your accessed instance.

Alt Image

Available information is:

  • Unique hardware ID on which permanent license is issued

  • License days left information for temporary or trial licenses

  • License type notifying if your product is licensed, under grace period or not licensed

  • Platform edition for which a license was registered

The window also contains version and build date for each licensed module.

A new license key can be pasted into License Key field and applied by pressing Add License Key button. An added license can either replace the current license, or provide additional functionalities like with add-on licenses.

The Clear license cache button changes the license type from CyberQuest Comunity to CyberQuest Enterprise Trial.

Click Alt Image to display detailed information about your license.

Alt Image

How Cyberquest license works

A short description of each licensing criteria can be found below:

  • Max. no. of users is the maximum number of active users the solution allows

  • No. of cores included: the solution runs on a standalone machine based on Linux Debian 9 operating system. The number of CPU cores included in license represents the maximum number of CPU cores of the system where Cyberquest runs

  • Up to no. of EPS included: is the maximum number of events per second the solution can process and is calculated mostly by the number of CPU cores, but also the hardware capabilities of the system

  • Max Data per day GB: depending on instance configuration, represents the maximum quantity of data the solution can process, according to your license.

  • Max number of CPU Cores per instance: is the maximum number of CPU cores which can be licensed for a Cyberquest instance. It can be increased by adding licenses for more CPU cores (add-ons)

  • Dashboards: notifies if the selected edition has access to Dashboards functionality in Cyberquest

  • Custom Dashboards: notifies if the selected edition has access to editing and creating dashboards by platform users

  • Max number of active Dashboards per Dashgroup: notifies on the maximum number of dashboards that can be active in a dashgroup for the selected edition

  • Max number of Dashgroups: represents the maximum number of groups which can be active in an instance

  • Data Permissions: number of filters: notifies on the permission to apply filters on data processed by Cyberquest, as well as the number of filters on which the licensing allows

  • Short Term Data Storage (days): depending on the storage capacity, represents the maximum number of days allowed by license, for which data can be stored in the online database

  • Long Term Data Storage (days): depending on the storage capacity, represents the maximum number of days allowed by license, for which data can be stored in the offline database, and if archiving is allowed by license

  • Max number of ElasticSearch Nodes per instance: is the maximum number of ElasticSearch nodes allowed by license to be attached to a Cyberquest instance

  • Max number of DTS Objects: is the maximum number of active objects allowed by license in Cyberquest's Data Transformation Services module.

  • Geolocation: notifies if the selected edition has access to geolocation functionality in Cyberquest

  • Reports: notifies if the selected edition has access to Reports functionality in Cyberquest

  • Compliance Reports: notifies if your license activates also the built-in compliance reports to standards

  • Asset Management: refers to inventorying capability in Cyberquest being active or not within your license

  • Windows Agent Collection: notifies if the selected edition can use Windows Agent-based gathering

  • ODBC Data Collection: notifies if the selected edition can use ODBC-based gathering

  • Syslog Data Collection: notifies if the selected edition can use syslog-based gathering

  • NetFlow Data Collection: notifies if the selected edition can use NetFlow gathering

  • NetFlow Stitching: notifies if the selected edition can pair send and receive events for TCP/IP communication and forming a unified event in this process

  • Multiple Data Server for advanced topologies: represents the ability to deploy multiple Data Server instances in order to collect data from various network segments

  • MultiTenancy: represents the ability to run multiple clients on the same instance

  • Simple Alerts (1 step rules): notifies if the selected edition can use, and the maximum number allowed, for single step alerts

  • Correlated Alerts (multi-step rules): notifies if the selected edition can use, and the maximum number allowed, for multiple step alerts

  • Summary Alerts: notifies if the selected edition can use, and the maximum number allowed, for summary alerts

Managing the Data Transformation Service (DTS)

Introduction to DTS

Cyberquest's Data Transformation Services is a general multipurpose data intelligence component used in treating and manipulating event data. DTS is a parsing service based on a script (JavaScript) that has a wide range of functions. Its primary role is performing additional transformations on data extracted from collected events. Typical usage involves extracting useful information from multiple fields, depending on several factors, when the source event does not split its useful information in to separate fields.

Data Transformation Service features: - prepares data for future easy access, allowing data to be transformed and loaded from heterogeneous sources; - used for event data correlation; - adds, modifies, removes, enriches and obfuscates partial or complete event data; - can remove an event; - can create multiple events out of an event; - dynamically creates and fulfills default set up lists with data collected from incoming events (for example, a list of Blacklist IPs) - can extract information from previously defined lists and take specific actions, such as sending alerts; - uses a built-in JavaScript engine, the administrator being able to engage out-of-the-box or create custom logic for performing various actions; - can send alerts; - encrypts and decrypts data; - can send HTTP/HTTPS requests; - can query data stored in events database; - and more.

DTS Service stores in memory lists of objects that the user can use to do specific tasks. For example, User logged on from specified IP ADDRESS list can be accessed by using the following script:

if(this.inputEvent.UserName == 'undefined')
{
    this.inputEvent.UserName = AD-whoisLoggedOn(this.inputEvent.SrcIP);
}

RealName (mapped by usernames and applications)

if(this.inputEvent.RealName == 'undefined')
{
    this.inputEvent.RealName = RealName(this.inputEvent.UserName);
}

NameLookups (by IP address )

if(this.inputEvent.SrcHost == 'undefined')
{
    this.inputEvent.SrcHost = NameLookup(this.inputEvent.SrcIP);
}

Generic Lists

if(this.inputEvent.EventType == 'undefined')
{
    this.inputEvent.EventType = genericListLookup(‘list_name’,this.inputEvent.PropertName);
}

Custom lists with dynamic objects can be created, which are shared between all components. The lists can be populated directly from DTS by using the "listRegister". To achieve this, use the following method:

if(this.inputEvent.EventType == ‘16’) //*16 means Failed Audit event*//
{
    listRegister(‘hosts_with_failed_audit’,this.inputEvent.Computer);
}

listRegister is defined as Void listRegister(String listName, String Value).

JS parsers (DTS objects)

Parsing or syntactic analysis is the process of analyzing a string of symbols, either in natural language or in computer languages, conforming to rules of a formal grammar. The parser's task is essential to determine if and how the input can be derived from grammar's start symbol.

JS Parser is a Javascript object that uses event logs and intelligently sorts data, making it easier for a user to interpret resulted information. Parsing is done by calling obj.exec with the event as a parameter in JSON Format.

JS parsers can be accessed from Web Interface by navigating to Settings > Rules > DTS Objects. DTS Objects page opens, listing defined objects. Here you can edit, delete or export a parser, and you can mark parsers as active or inactive. The Actions menu includes options for importing an object or creating a new DTS object from scratch.

Alt Image

To export a parser, press Alt Image button next to it. The export is saved as a proprietary CQO file. Likewise, to import a parser select Alt Image in Actions menu.

To edit details for a specific object, press Alt Image button next to it. Edit DTS Objects page opens allowing you to change the Name and Description, correct the Script or enable/disable the object.

To delete a parser from the list, press Alt Image button next to it. As a measure of precaution, you will be asked to confirm deletion.

Creating a new JS parser

In DTS Objects page, select Alt Image from Actions menu. Add DTS Object configuration page opens allowing you to create the script for a new parser and mark it as active or inactive. When you finished creating the parser, press Alt Image button to save changes.

Here is a sample JS parser:

Alt Image

Filter rules

Cyberquest uses an intelligent event filter mechanism for sending data to JS parsers. You can instruct DTS on how to filter events sent to parsers by creating filter rules.

Filter rules can be managed by navigating to Settings > Rules > Filter Rules. Filter Rules page opens allowing rules to be defined based on operators like "eq", "noteq", "isInList", "isNotInList", "startsWith", "endsWith", "intInterval". There is practically no limit to adding additional fields.

You can edit, delete or export a rule, and you can mark rules as active or inactive. The Actions menu includes options for importing or creating a new filter rule from scratch.

Alt Image

To export a rule, press Alt Image button next to it. The export is saved as a proprietary CQO file. Likewise, to import a rule select Alt Image in Actions menu.

To edit details for a specific rule, press Alt Image button next to it. Edit Filter Rule page opens allowing you to change the Name and Description, as well as add, correct or delete already defined filters.

To delete a rule from the list, press Alt Image button next to it. As a measure of precaution, you will be asked to confirm deletion.

Creating a new filter rule

In Filter Rules page, select Alt Image from Actions menu. Add Filter Rule configuration page opens allowing you to create a new rule.

Alt Image

A rule can have one or more filters defined:

  • In Select Field drop-down list, select the event field you want to filter

  • In Operator drop-down list select from one of the available operators eq, noteq, isInList, isNotInList, startsWith, endsWith, intInterval

  • Type in the Value used for comparison

By default, the new rule will be active. Press on Alt Image switch to inactivate the rule.

At any moment you can press Alt Image to delete a filter.

When you finished creating the rule, press Alt Image button to save changes and return to rules list.

Data acquisition rules (DA Rules)

Cyberquest includes a decisional engine based on rules that allows to combine filter rules with JS parsers for the purpose of establishing a granular data collection flow. As such, for a given flow of events, you will be able to assign one or more parsers to specific filtering rules and this way instruct the platform on what events will be collected.

Data acquisition rules can be accessed from Web Interface by navigating to Settings > Rules > DA Rules. DA Rules page opens, listing defined rules. Here you can edit, delete or export a rule, define the order in which rules are applied, and you can mark rules as active or inactive. The Actions menu includes options for importing or creating a new acquisition rule from scratch.

Alt Image

To export a rule, press Alt Image button next to it. The export is saved as a proprietary CQO file. Likewise, to import a rule select Alt Image in Actions menu.

To edit details for a specific rule, press Alt Image button next to it. Edit DA Rule page opens allowing you to change the Name and Description, as well as add, correct or delete already defined filters.

To delete a rule from the list, press Alt Image button next to it. As a measure of precaution, you will be asked to confirm deletion.

To change the order in which rules are applied, simply drag and drop. Please note the last rule in list applies last and therefore may supersede other uphill rules.

Creating a new DA rule

In DA Rules page, select Alt Image from Actions menu. Add DA Rule configuration page opens allowing you to create a new rule.

Alt Image

Below described attributes are similar to those found in Edit DA Rules configuration page:

  • In Name field, type a name that identifies the newly created data acquisition rule. This name will appear in DA Rules list

  • In Description field, insert an explanatory rule description

  • AND filter rules allow the usage of AND operators in order to get information on multiple layers

  • OR filter rules allow the usage of OR operators in order to get information on multiple layers

  • Data Storages allows you to select the long-term data storage to use

  • In DTS Objects drop-down list, select the DTS objects for which this rule will apply

  • In Order field, set a usage priority for the newly created data acquisition rule

When you finished creating the parser, press Alt Image button to save changes.

Working with Data source status feature

To verify all data collection status from all sources that send events to Cyberquest or sources collected by Cyberquest, the tool provides a dedicated status screen.

In Web Interface select Settings > Data source status. Data Sources Status page opens, listing all data sources collected by Cyberquest.

The collection status is shown in color code for each data source. Available statuses are:

  • Disabled

  • Collecting

  • Stopped or critical error

  • Waiting for next collection

An Alt Image icon present signifies that collection is scheduled to execute at defined time intervals, while all others are executing in real time.

At any time, you can sort the list by any of the columns, or you can export the list by pressing Alt Image button.

Alt Image

It is important to note here that due to the large number of data collections Cyberquest can support, the collection status list can grow very long.

You can choose to display up to 100 entries per status page. Please remember not to combine a large number of entries with automated page refresh, to avoid a decrease in performance.

The columns menu at the top of the page allows you to choose which columns are displayed for all entries in list. These are described in the table below:

Field Description
Computer Name Source name (network IP address or resolved FQDN)
Log Name Name of the log source
Type Log type
Messages Number of collected events
Last Received Time Last current time when data was received from source
Last Local Time Last device time when data was received from source
Last Update Time Last time a modification was made for data source
Last Message Last message from data collector
Last Error Last error message from data collector
Next Collection Date and time when next collection will start
Producer Module or agent that collected the events
Producer Uptime Uptime of module or agent that collects events
Extra Data Comments
Alert Interval Minutes Time interval to check source status

Managing Event Dictionary

Working with event definitions

Cyberquest ships with a full event dictionary built around Windows operating systems. The dictionary is under continuous expansion, and future platform releases will start including event dictionaries for all major supported technologies.

A list of all events available at the time of editing this document can be found in Appendix: Event Dictionary.

Unlike other SIEM solutions on the market, Cyberquest's dictionary is open, which means at any time you can edit, export and delete existing event definitions, or create and import new ones -- building your own dictionary supporting technologies you have under management.

The event dictionary can be accessed from Web Interface by navigating to Settings > Event Dictionary. The page opens, listing defined objects. Here you can manage existing definitions and from Actions menu, import an object or create a new definition from scratch.

Alt Image

To export a definition, press Alt Image button next to it. The export is saved as a proprietary CQO file. Likewise, to import a definition select Alt Image in Actions menu.

To edit details for a specific object, press Alt Image button next to it. Edit event definition window opens allowing you to change the Name and Description, correct the Script or enable/disable the object.

To delete an event from the list, press Alt Image button next to it. As a measure of precaution, you will be asked to confirm deletion.

Events can be searched in the search bar by event ID, event name, or its description.

Alt Image

Creating a new event definition

In Event Definitions page, select Alt Image from Actions menu. Add Event Dictionary configuration page opens allowing you to create the new definition.

Alt Image

All fields are free text, which permits complete freedom on defining a new event. The template contains up to 150 custom fields to add. As a general recommendation, it is advisable to define a company-wide standard for issuing EventIDs, event names and platforms for all the applications in scope.

When you finished creating the parser, press Alt Image button to save changes.

Managing Jobs

Working with jobs

You can create and use jobs to act on data that is parsed and stored in the online or archive repositories. Since the volumes in question can become very large over time, and due to inherent manipulation restrictions for data in use, or data in different stages of evolution, simple copying, moving or deletion of storage files is not recommended.

Go to Settings > Jobs > Jobs.

Jobs page opens allowing you to see the list of already defined jobs or create a new one. You will notice there are no pre-defined jobs in place.

Alt Image

You can create as many jobs as you want, with the only consideration to the load their execution will produce on the solution environment. Jobs can be very resource intensive for your server's processor, memory, storage and possibly network.

In the list of already-defined jobs, you are provided with details like the type of job, the server where is executed, and if it is active or not. Explanations on these details are given in section dedicated to creating a new job.

For each job in the list, you can choose to edit, delete or execute the job. Click Alt Image to edit the job. Edit Job page opens with configuration options similar to creating a new job. Perform any needed changes and click Save to save and close the page.

Alt Image

Click Alt Image to execute a job. Job executions can be checked in Job Executions page described in the following section.

To delete a job from the list, press Alt Image button next to it. As a measure of precaution, you will be asked to confirm deletion.

Managing job executions

Go to Settings > Jobs > Job Executions. Job Executions page opens allowing you to see the history of jobs that were executed, their result and also the summary of jobs that are currently in execution.

Alt Image

Deleting an execution from this list produces effects only if the job was not finished yet. As a general rule, it is advisable to wait for an execution to finish before deleting.

Creating a new job

Go back to Settings > Jobs > Jobs. In Jobs page, select Alt Image in Action menu. Add Job configuration page opens allowing you to create a new job.

Alt Image

Below described attributes are similar to those found in Edit Job configuration page. Please note the available options will change depending on the job type selected:

  • In Name field, type a name that identifies the newly created job. This name will appear in Jobs list.

  • In Description field, insert an explanatory job description.

  • Job Type drop-down allows you to choose between:

  • Import job: to import events from a source repository to an online repository or to an archive. Import jobs are very useful for consolidation operations, or when decommissioning obsolete repositories.

  • Delete job: to delete events from a repository. Deletions may be needed to free up space that is used by events from obsolete applications, or as a result of an import.

  • Copy job: allows you to copy data from one data storage to another. This operation is only possible between offline repositories and allows for data duplication

  • Right to be forgotten: is a special type of job allowing granular deletion of personal data based on filters. These jobs were introduced for compliance with data protection regulations.

  • Data Storage is always the source data storage, no matter which type of job you choose

  • Where to Import option for import jobs, can be either an online or offline repository where the import takes place

  • Period Spec. Type drop-down allows you to choose if the job will act on data based on a selected time interval or the most recent number of units, where a unit can be second, minute, hour, day, month or year.

  • Start Date and End Date allow you to define the time interval for which job execution will action.

When you finished creating the job, press Alt Image button to save changes.

Managing Dashboards, Filters and Objects

Managing dashboards

Dashboards page allows you to granularly configure dashboards appearance and behaviour in Dashboards module. To access the page, go to Settings > Management > Dashboards. All objects in your Cyberquest instance are listed here.

Alt Image

Dashboards can be exported Alt Image and imported Alt Image, edited Alt Image, or deleted Alt Image.

To create a dashboard, press Alt Image button. A window will open that allows for dashboard configuration:

Alt Image

Save Dashboard window opens.

Press Save to save your changes and close the window, or Cancel to close the window without saving.

Managing filters

Filters page allows you to modify predefined filters or create new ones. To access the page, go to Settings > Management > Filters.

Alt Image

To edit and existing filter, or create a new one, press Alt Image or select Alt Image in Action menu. Edit Filter configuration page opens.

Alt Image

All predefined filters have queries built on compliance standards. Editing these usually involves advanced knowledge on building queries. As a general recommendation, it is advisable to always create a new filter based on an existing one and test before introducing to production.

When you finished creating or editing the filter, press Alt Image button to save changes.

Managing objects

Objects Management page allows you to modify predefined objects or create new ones. To access the page, go to Settings > Management > Objects.

Alt Image

Anything can be an object: users, computers, IP addresses, an IP address range, network equipment and so on. Most objects are created automatically. For example, when logging in with a new Windows domain account, the correspondent object is also created.

New objects can be created also manually, or by importing from a CSV file. Once added to the system, they can be edited by pressing Alt Image . The list of editable attributes is limited (name, value, corresponding object list). Their role in the platform is to provide the needed display consistency in lists, making easier for an administrator to correctly identify the target of their investigations.

Application Settings

Application settings overview

Cyberquest's Web Interface includes the administrative section needed for a visual configuration of your audit system. This is done under Settings > Application Settings. The administrator will be presented will a distinct section listing all configurable components, some of them already being discussed in previous chapters.

Alt Image

Customizing the Web Interface

Select Customize entry to access the instance customization page. You can change the following:

  • Company email disclaimer

  • Company logo

  • External reporting server

  • License server (by default, local server)

  • Number of login attempts before the user account is blocked

  • Login welcome message to be shown at logon

  • Send to external link

Alt Image

Adjusting your Cyberquest environment

Select Administration entry to access the instance administration page. Here you can change all entries that are explained in sections dedicated to Cyberquest configuration files.

Alt Image

Adjusting data acquisition settings

Select DataAcquisition entry to change data acquisition settings. Here you can change all entries that are related to data aquisition.

Alt Image

Adjusting data correlation settings

Select DataCorrelation entry to change data correlation settings. Here you can change all entries that are related to data correlation.

Alt Image

Adjusting data server settings

Select DataServer entry to change Data Server Service settings. Here you can change all entries that are related to Data Server.

Alt Image

Adjusting data storage settings

Select DataStorage entry to change data storage settings. Here you can change all entries that are related to data storage.

Alt Image

Defining data storages

Navigate to Settings > Data Storages. In this tab we can use the following options to administer the module:

  • Create new Data Storage

  • Edit existing Data Storage

  • Delete existing Data Storage

  • Selecting default Data Storage-site

Alt Image

To create a new data storage, select Alt Image in Actions menu. New Data Storage configuration page opens. Fill in the path, server, change to "Active" status save by clicking Submit.

To use a specific data storage as default, in Data Storages page click on the check mark next to your target. The "IsDefault" status will change in \"Yes\".

Adjusting ElasticSearch settings

Select ElasticSearch entry to change NoSQL settings. Here you can change all entries that are related to ElasticSearch nodes and engine.

Alt Image

Adjusting email settings

Select Email entry to change email settings. Here you can change all entries that are related to email sending and receiving.

Alt Image

Adjusting reports export settings

Select ReportsExport entry to change export setting for your reports. Here you can change all entries that are related to exporting reports.

Alt Image

Adjusting retention time

Select RetentionPeriod entry to change the retention period of stored data. Here you can change all entries that are related to retention.

Alt Image

Managing Windows Gathering Agent

Manage Cyberquest Log Gathering configuration files

In order to properly configure Cyberquest Log Gathering Agent to choose the type of logs, computers from where logs will be collected and where to send collected logs, the following files need to be edited:

  • Agent.exe.config (default location is: C:\Program Files (x86)\Cyberquest LogAgent)

  • Collections.xml (default location is: C:\Program Files (x86)\Cyberquest LogAgent)

These files are initially configured when the agent is first deployed. However, changes in your audited environment may require adjustments from time to time.

Please note that in order to successfully change entries in configuration files, the agent service must be in stopped state:

a. Using an administrative account, authenticate to the Windows log gathering computer having the agent installed and open Services management console

b. Stop Cyberquest Log Gathering Agent service.

c. After finished performing changes, start Cyberquest Log Gathering Agent service.

Data-Storage

Allows for advanced configuration of data storages used by Cyberquest. To edit Data-Storage, open /var/opt/Cyberquest/datastorage/conf.xml file on Cyberquest server.

You can find all configurable variables in the following table:

Parameter Description
maxEventsPerFile Specifies the maximum number of events allowed per stored file
fileWriterTimeout Specifies the timeout interval for the event writer
mqUserName Specifies the administrative username for MQ service access
mqPassword Specifies user's password for MQ service
mqHost Specifies the MQ service server. In distributed architectures, it may differ from the default Cyberquest server
mqVhost Specifies the MQ service virtual server. In distributed architectures, it may differ from the default Cyberquest server
mqPort Specifies the network communication port used by MQ service
mqExchangeName Specifies the exchange name used by MQ service
mqQueueName Specifies the MQ queue name
mqReceiveQueueType Specifies the MQ Receive queue type
mqRouting Specifies the routing path for message queues
mqReceiveCommandExchangeName Specifies the MQ Receive command exchange name
mqReceiveCommandQueueName Specifies the MQ Receive command queue name
mqReceiveCommandQueueType Specifies the MQ Receive command queue type
mqReceiveCommandRouting Specifies the MQ Receive command routing path
mqSendExchangeName Specifies the MQ Send exchange name
mqSendQueueName Specifies the MQ Send queue name
mqSendRouting Specifies the MQ Send routing path
mqSendQueueType Specifies the MQ Send queue type
serverGuid Specifies the unique server GUID
encryptionPublicKeyFilePath Specifies the file path for defined public key
encryptionPrivateKeyFilePath Specifies the file path for defined private key
dbDriver Specifies the database driver used for SQL processing
dbUserName Specifies the administrative username used to access the database
dbPass Specifies user password used to access the database
dbUrl Specifies the database access URL

Data-Acquisition

Allows for advanced configuration of Cyberquest's Data Acquisition Service. To edit Data-Acquisition, open /var/opt/Cyberquest/dataacquisition/conf/config.ini file on Cyberquest server.

You can find all configurable variables in the following table:

Parameter Description
CommandPort Specifies the network port used by server address where Analyzer commands are sent
AnalyzerPort Specifies the network port used by server address where Analyzer training information is sent
AnalyzerAddress Specifies server's network address where Analyzer events are sent
Config_DB_HOST Specifies the hosting server for internal MySQL database
Config_DB_USER Specifies the administrative username for accessing the internal MySQL database
Config_DB_PASSWORD Specifies user's password for the internal MySQL database
Config_DB_DB Specifies the internal database used by server
EL_Url Specifies the ElasticSearch engine URL
EL_Port Specifies the ElasticSearch engine access port
FIFO_size Specifies a maximum size for the inner collection list
BUFFER_size Specifies the number of events sent in a single burst to FIFO queue
HTTP_SERVER_PORT Only for internal use. Default network port is 8082
UDP_SERVER_PORT Specifies the UDP port used by DAS hosting server
SYSLOG_UDP_SERVER_PORT Specifies the network port used to forward syslog data using UDP protocol
LIC_PATH Specifies the license file path on server
CLEANUP_CRON Specifies the retention period clean-up frequency
ARCSIGHT_UDP_SERVER_PORT Specifies the network port used to forward CEF data using UDP protocol
no_of_threads Specifies maximum number of threads. This field auto-fills
debug_level Changes verbosity level. Default level is 2
RMQ_host Specifies the messaging queue server. In distributed architectures, it may differ from the default database server
RMQ_username Specifies the administrative username for queuing services
RMQ_password Specifies user password for queuing services
RMQ_queue Specifies the messaging queue name for queuing services
maxmindb_path Specifies the server path for "maxmin" database
run_collection_servers Indicates true/false flag for cluster type deployments

Horizontal scaling through NOSQL service

Horizontal scaling is a strategy that Cyberquest users can use to enhance the performance of the server node by adding more instances of the server to your existing pool of servers so the load can be equally distributed. In horizontal scaling, the capacity of the individual server is not changed, but the load on the server is decreased. Horizontal scalability is achieved with the help of a distributed file system, clustering, and load balancing. Some of the reasons why businesses choose to scale horizontally include an increase in their I/O concurrency, need to reduce the load on existing nodes and to expand disk capacity. Horizontal-scaling is considerably easy as you can add more machines to the existing pool. It follows the partitioning of the data in which each node contains only one part of the data. For the NOSQL sevice that ensures horizontal scaling, Cyberquest uses ElasticSearch.

Introduction to ElasticSearch Clustering

ElasticSearch is built to be always available, and to scale with your needs. Scaling can come from deploying more performant servers, especially on CPU and memory considerations (vertical scale, or scaling up) or from deploying more servers (horizontal scale, or scaling out). While ElasticSearch can benefit from more powerful hardware, vertical scale has its limits. Real scalability comes from horizontal scale, the ability to add more nodes to the cluster and to spread load and reliability between them. With most databases, scaling horizontally usually requires a major overhaul of your application to take advantage of these extra boxes. By contrast, ElasticSearch is distributed by nature: it knows how to manage multiple nodes to provide scale and high availability. This also means that your application doesn't need to care about it. A node is a running instance of ElasticSearch, while a cluster consists of one or more nodes in the same cluster.name, that are working together to share their data and workload. As nodes are added to or removed from cluster, the cluster reorganizes itself to spread the data evenly.

Advantages of ElasticSearch clusters are:

  1. Distributed data: In cluster data is distributed, replicated to another server. So, in case of failure of one node, data can be restored from replica node. It avoids single point of failure.

  2. Dedicated node roles: Every node has dedicated role assigned to it, that ensures specific role and role-based load distribution hence increasing performance. Here are two important node roles

    • Data node: These nodes only store data and do data related operations, search and data manipulation.

    • Master node: Master of all nodes, it holds responsibility of overall cluster, addition and removal of nodes from cluster, keeping track of alive nodes, master reselection in appropriate cases.

  3. Scalability: Cluster model is easily scalable to multiple no of nodes, thus increasing performance and reliability of ElasticSearch.

One node in cluster is elected to be the master node, being in charge of managing cluster-wide changes like creating/deleting an index or adding/removing a node from the cluster. The master node does not need to be involved in document-level changes or searches, which means that having just one master node will not become a bottleneck as traffic grows. Any node can become the master.

Every node knows where each document lives and can forward our request directly to nodes that hold the data we are interested in. Whichever node we talk to manages the process of gathering a response from the node or nodes holding data and returning the final response to client. All this is managed transparently by ElasticSearch.

Cyberquest takes advantage of this technology so whether the underlying database is clustered or single node deployment, no additional configuration of Cyberquest is required.

Checking cluster health

Cyberquest comes installed with the ElasticSearch Cerebro service for a visual representation of the database, by accessing the Cyberquest IP address on port 9000 via a web browser.

The results are:

Alt text

for a single node installation, or:

Alt text

for two or more clustered nodes.

Adding cluster nodes

Out-of-the-box, ElasticSearch is configured to use unicast discovery to prevent nodes from accidentally joining a cluster. Only nodes running on the same machine will automatically form a cluster. To use unicast, you provide ElasticSearch with a list of nodes that it should try to contact. When a node contacts a member of the unicast list, it receives a full cluster state that lists all the nodes in cluster. It then contacts the master and joins the cluster.

This means your unicast list does not need to include all the nodes in your cluster. It just needs enough nodes that a new node can find someone to talk to. If you use dedicated masters, just list your three dedicated masters and call it a day. This setting is defined in elasticsearch.yml configuration file:

Alt text

discovery.zen.ping.unicast.hosts: [\"OtherEasticSearchHost1\",\"OtherEasticSearchHost2\"]

When finished, save and restart ElasticSearch service:

systemctl restart elasticsearch.service

Additional ElasticSearch documentation

Additional database documentation can be found here: https://www.elastic.co/guide/en/elasticsearch/guide/master/index.html

Backing Up Cyberquest

MySQL database dump

Most of application configurations for Cyberquest is stored in the internal MySQL database.

The database is backed up using a MySQL dump script:

DATE=date +%Y-%m-%d

mkdir -p /data/mysqlbackups/

mysqldump -u [dbuser] -p[dbpass] --all-databases | gzip > /data/mysqlbackups/$DATE.sql.gz

Replace [dbuser] and [dbpass] with MySQL username and password respectively.

This script is executed daily on a cronjob basis.

To check if the script is added to crontab scheduler use the following command: crontab -l

Alt text

The following line needs to be present:

Alt text

If it is not, then it can be added with: crontab -e

a. Paste the line on last row:

30 2 *** /var/opt/Cyberquest/dataacquisition/bin/mysql_full_backup.sh

b. Save and exit

c. Finally, check /data/mysqlbackups/ folder for backed up database presence

Event data backup

By default, every message collected by Cyberquest is automatically sent to data storages, as configured in Settings > Data Storages page:

Default: /data/storage/default

Alt Image

Stored events are normalized in JSON format, compressed, encrypted and digitally signed.

They can be later imported to a backup storage. In order to achieve that, follow these steps:

a. Create a new data storage as described in chapter Managing Repositories | Configuring data storages

b. Create an import job as described in chapter Managing Jobs | Creating a new job

c. Select the newly created data storage as target

Integration with third-party backup solutions

In either deployment configuration (physical appliance, virtual appliance, installable software), Cyberquest allows external connections to be set for data extraction and backup.

The appliance is built on Linux Debian operating system, and each platform layer can be backed up following vendor practices for:

For virtual appliances, you can use industry best practices defined for VMware or Windows Hyper-V.

Note: Although there are no registered cases with third-party agents interfering with Cyberquest's processes, it is recommended to use an agentless tool for backing up virtual appliances. Many tools on the market, like VEEAM or Quest Rapid Recovery, use an agent installed on hypervisor to handle virtual machines, therefore installing nothing on the virtual machine itself. However, if you need to deploy a backup agent within the virtual appliance, please address Nextgen Software support team for any issues with platform's native processes.

Default backup locations for 3rd party backup solutions are:

a. Configuration database settings Default: /data/mysqlbackups/ folder

b. Event data backup Default: /data/storage/default/ folder

c. The current Cyberquest/Cyberquest installation files: Default: /var/opt/Cyberquest/ folder