Page Comparison

...

• Development Environment

  • This integration is developed external to the platform using a Python 3.6 environment and the TcEx Framework. Please see the Getting Started section for specific details of how to configure your environment.

  • The latest version of the TcEx v2.0 framework should be used. Specifically, your requirements.txt should look like this: tcex>=2.0.0,<2.1.0

    • See this set of examples for the TcEx v2.0 framework.

• Architecture

  • The template job_batch from the TcEx Framework should be used as a starting point for this integration. In particular, the examples in the template for how to submit data, log messages, and other configurations should be used.

  • Incoming indicator feeds must make use of the TcEx Batch module for data manipulation.

    • See this error guide for common errors you might see using the Batch module.

  • Supporting differential data updates is optional but desired. Differential data updates is the process whereby only changes are sent regularly after an initial synchronization.

    • See References - Persistent Input Values for details on how to store a timestamp from run-to-run in your integration.

  • The property self.tcex.session_external should be used to retrieve a Python requests object configured with the ThreatConnect proxy settings.

    • This should be used for external HTTP communication to ensure that calls work properly regardless of the environment. The standard Python requests library should not be used.

    • For TcEx v1: When using this method, it is recommended that you specify a ‘user-agent’ header in order to identify your integration on remote systems. This should include the name and current version of your app.

      • You can make use of the self.tcex.ij.display_name and self.tcex.ij.program_version parameters to dynamically populate this field.

• Configuration

  • The owner of the data being imported must be configurable. This input parameter must be named tc_owner.

    • When you’re testing in the PartnerStage environment, you must use your Source for this field. This should be <Organization> Source where <Organization> is your company name. For example, if your company name is SecuLast, this would be SecuLast Source.

  • The log-level for the job must be configurable. The input parameter must be named tc_log_level and should be a choice. See References - tc_log_level Input Parameter Configuration for an example.

  • A Feed Deployer configuration should be included in the Runtime App after your initial build is provided to your Solutions Engineer for in-platform testing.

    • You cannot include Feed Deployer in your initial build for in-platform testing because the steps required to create the Feed Deployer configuration require having a built in-platform already.

    • See References - Feed Deployer Configuration Example for specific steps.

• Data Mapping

  • Follow the guidance provided in our References - Threat Intelligence Data Mapping document.

  • Only standard indicator types should be used in this feed. You may not implement custom Indicator types of any kind.

    • NOTE: The PartnerStage environment does not contain any custom indicators.

  • Threat Rating and Confidence must be set according to the ThreatConnect best-practices. If the incoming data source does not provide this information, these values must remain configurable and should default to 0.

    • NOTE: We recommend against putting a blanket confidence or threat rating on all indicators from a source. Instead, provide a configuration parameter for users to configure this behavior only if necessary.

  • The tag scheme used should only contain the bare words associated with the tag and should not reference the name of the source providing this information.

    • NOTE: The name of the Owner for the source will provide the appropriate name context.

  • Victim and Victim Asset data is not a candidate for contribution via Threat Intelligence Feed at this time. See Integrations - Low-Volume Alerts Processing Description for an alternative.

• Miscellaneous

  • Calls for a timestamp should refer to the utilities provided within the TcEx framework.

  • See References - Application Logging Guidelines for specific details around logging.

  • The deletion of invalid indicators is optional but desired. If this logic is not available within the integration itself, recommendations on the appropriate Indicator Deprecation rules must be supplied with this integration.

    • An example of this configuration can be provided.

  • The management of differential data (deltas) should be done within your integration on the ThreatConnect platform.

    • This allows an easy reset of this control mechanism if this is desired without requiring a change in a remote system which could lead to unexpected results.

    • See References - Persistent Input Values for how to store data run-to-run.

  • The management of any subscription-specific parameters for premium intelligence must be managed by the Partner’s API. This is typically done through unique API keys. The ThreatConnect Platform does not perform any third-party subscription management for feeds.

  • When making use of the TcEx Framework, it is not necessary to implement any batch controls to limit the number of indicators in a TcEx Batch. The framework will automatically chunk, submit, and monitor batches on your behalf.

    • NOTE: When you anticipate large amounts of data, it is recommended to implement the Batch.save() method after each object in order to temporarily write data to the disk and reduce memory use.

  • Batch submissions should be checked for fatal errors. See this link for an example.

    • NOTE: You must check your integration for errors and remediate any problems found prior to submitting your integration for vetting.

  • Exit codes should follow Job App Exit Codes.

• Distribution

  • This integration will be distributed by GitHub. The Partner should publish the integration assets to their own GitHub. Once published, the Partner can submit a pull request for the ThreatConnect GitHub to have the asset published in this repository.

    • Please create a folder name that reflects your integration’s name (based on the Integration Naming guidelines).

...