Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Current »

Overview

This document is intended for a technical audience and provides a detailed description of Low-Volume Alerts Processing with the ThreatConnect Platform. Low-volume alerts (less than 1000/alerts/day) processing integrations make use of the in-platform Playbooks by way of a Playbook Template to model data handling. Typically, these integrations ingest alerts from remote systems, enrich the provided alert information using the ThreatConnect Platform, and make a decision on whether extended data is stored and users notified for review.

Integration Attributes

  • Data Directionality: Inbound - An alert configured in a remote system is fired via Webhook or other mechanism to an incoming listener configured on the ThreatConnect Platform.

  • ThreatConnect Technologies:

    • Playbooks (Templates)

    • Playbook Components

    • Playbook Apps (App Service)

  • Execution:

    • The Playbook Template is typically configured with an HTTP trigger to ingest incoming data.

  • Deliverables:

    • Playbook Template (to describe your entire alert-processing scenario)

    • Playbook App or Component (to provide assistance in processing alert data if necessary)

Example Integration Diagram

Diagram Explanation

  1. An event is created in a remote platform.

  2. Event data is appropriately filtered in the remote platform such that the volume fired at the ThreatConnect Platform remains specific and low-volume.

  3. The event payload is prepared in the remote system.

  4. The event payload is fired as a Webhook towards a receiver in the ThreatConnect Platform.

    1. This receiver could be an HTTP Trigger Playbook (Template) or an App Service (which is part of a Playbook Template). The decision on what is used here is best discussed with your ThreatConnect Solutions Engineer.

  5. The event is received in the ThreatConnect Platform as part of a Playbook Template.

  6. Data is processed from the event (JMES Path, etc).

  7. (Optionally) Additional enrichment is requested (such as requesting a document or other body of data regarding the received event).

    1. This is typical when the full context of an event cannot be supplied via the trigger itself.

    2. The work for the additional enrichment should be completed as a separate On-Demand Enrichment integration.

  8. (Optionally) The enrichment request is received by an enrichment source.

  9. (Optionally) The payload of enrichment data is prepared.

  10. (Optionally) The enrichment response is provided.

  11. All enrichment data is processed and typically a decision is made about whether or not to proceed based on known information.

    1. For example, do we have any reason now to continue with these indicators based on other data that we have? If not, we should discontinue. If so, we’ll proceed.

  12. We store the enriched data in the platform (Indicators along with Groups as necessary and according to data modeling).

  13. We fire an in-platform notification to notify ThreatConnect users of information that should be reviewed.

  14. The Playbook logic ends.

Integration Key Points

  • Development Environment

    • Playbooks Templates or Components: This integration is a codeless path and is developed in-platform using the Playbook Designer. Please see this KB article on Playbooks for more information.

    • Playbook Apps: This integration is developed in-platform using the App Builder. Please see this KB article on App Builder for more information.

      • It is possible to develop an implementation outside of the platform using a traditional development environment. Please discuss this with your Solutions Engineer if you feel this appropriate for your project.

      • The latest version of the TcEx v2.0 framework should be used. Specifically, your requirements.txt should look like this: tcex>=2.0.0,<2.1.0

  • Architecture

    • General:

      • Output variable names should follow the schema defined in the References - Playbook Output Variable Naming Schema article.

      • Playbook Apps provided with this type of integration are specifically designed as App Services triggers.

        • If additional enrichment is necessary to complete the Playbook Template scenario for your integration, a separate but similar integration of On-Demand Enrichment should be planned.

      • A main goal of this integration is to show the ability to enrich data using the ThreatConnect Platform. Therefore, CAL or other enrichment data from the ThreatConnect Platform should be used in Playbook Templates that are provided.

      • Data should be filtered/processed prior to being shipped to the ThreatConnect Platform. The ThreatConnect Platform should not be used for bulk data processing.

      • This integration type is intended for low-volume applications where less than 1000 alerts per-day are expected on a regular basis.

      • As part of your overall solution, you may be developing multiple integrations. Your path for work should complete items in the order listed below:

        • Develop all Playbook Apps

          • This is important to ensure that the structure of the app is consistent and to ensure that a release version of your app is used in Playbook Templates.

          • This treats the Playbook App as a single unit and can be released on its own.

        • Develop all Playbook Components

          • This is important to ensure that the structure of your component is consistent when used in Playbook Templates.

        • Develop all Playbook Templates

          • You must use release versions of Apps in your final work and therefore those must be completed.

    • Playbook Apps:

      • The property self.tcex.session_external should be used to retrieve a Python requests object configured with the ThreatConnect proxy settings. This should be used for external HTTP communication to ensure that calls work properly regardless of the environment. The standard Python requests library should not be used.

        • For TcEx v1: When using this method, it is recommended that you specify a ‘user-agent’ header in order to identify your integration on remote systems. This should include the name and current version of your app.

          • You can make use of the self.tcex.ij.display_name and self.tcex.ij.program_version parameters to dynamically populate this field.

  • Configuration

    • General:

      • Your integration should accept variable inputs for all parameters (as opposed to any hard-coding of inputs).

    • Playbook Components and Apps:

      • Errors: By default, an error in a Playbook will result in the orange path being followed as an error. A non-zero exit code from a Playbook app is considered a failure. Providing an input of Fail on Error allows the opportunity to continue the blue path with a fail and success output count is an alternative and allows the Playbook creator to control the exit behavior.

        • The exit code 1 should be used when exiting due to a failure.

      • Fail on No Results: By default, returning no results should result in the orange path being followed as an error. Providing an input of Fail on No Results allows the opportunity to continue the blue path with an output count of how many items were returned is an alternative and allows the Playbook creator to control the exit behavior.

      • Verifying SSL Certificates: By default, all SSL certificates should be validated. Providing an input of Verify SSL Cert allows the opportunity for the user to decide to disable validation if required for private APIs only. If this option is used, documentation must also clearly state that using validation is a best practice and that disabling this is only appropriate during testing.

        • NOTE: This option cannot be used for services that hit public APIs as these must be secured appropriately.

  • Data Mapping

    • Data brought into the ThreatConnect Platform should be mapped in an exercise with your ThreatConnect Solutions Engineer.

    • Data should not be stored using Playbook Apps. Storage should only be part of the Playbook Template or Component functionality.

    • Indicators that are ingested as potentially malicious (but subject to further investigation) should be inserted into the platform with the following characteristics:

      • Inserted into a specific owner for housing this data

        • NOTE: This should not be shared with an owner that also contains malicious Indicator records.

      • A Threat Rating of 1 skull (to indicate that it is suspicious)

      • A Confidence of 50 (to indicate that it is possibly malicious)

  • Miscellaneous

    • General:

      • You must check your integration for errors and remediate any problems found prior to submitting your integration for vetting.

    • Playbook Apps:

      • Make use of the Labels feature to provide a reasonable number of search terms to help people find your integration within the Platform. These labels might be terms associated with your functionality but not included in the name of your integration. These labels should be specific in nature.

      • Calls for a timestamp should refer to the utilities provided within the TcEx framework.

      • See the References - Application Logging Guidelines for specific details around logging.

      • See the documentation on some of the built-in decorators that we provide to simplify processing in your code.

      • Make use of the App Note section in the project Metadata tab in App Builder to provide general context for your integration. This should also include a link to your User Documentation as it is displayed in-platform when your component is used.

      • Make use of the Help Note fields for each input to describe the purpose and expected format(s) for the input.

        • In particular, the Help Note for your Action field should contain a description of each of the actions at a high-level.

      • Keep all in-platform notes/descriptions as concise as possible. Reserve extended explanation to your User Documentation.

    • Playbook Components:

      • Make use of the Description section in the Component definition to provide context for your integration. This should also include a link to your User Documentation as it is displayed in-platform when your component is used.

    • Playbook Components and Templates:

      • Follow best-practice guidelines when creating Playbook Components and Templates:

  • Distribution

  • No labels