References - Batch Module Error Guide
- Tianyi Chen
- John Hall (Unlicensed)
The ThreatConnect TcEx Framework provides the batch module to create, delete, and update both Groups and Indicators in the ThreatConnect platform. The App developer can dynamically build data objects, and the Batch module will write the data to the ThreatConnect API. When writing to the ThreatConnect API, errors can occur and this guide will help understand common batch errors.
All Batch Errors and Short Descriptions
Error Code | Short Description |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Common Batch Error Explanations
Error Code 0x1002 - Permission Error
This error generally occurs when the user does not have the correct permissions to perform a certain action.
For example a user only has contributor permissions to a source. A contributor does not have permissions to delete indicators or groups. Editor permissions would be needed to delete.
Error (0x1002): You are not authorized to delete group attributes.Error Code 0x1003 - JsonSyntax Error
This error generally occurs when there is an error in the JSON syntax.
For example, the eventDate field incident group had a typo which caused it to not be formatted to the ISO 8601 standard.
Error (0x1003): Encountered an unexpected Exception while processing batch job. Last
known JSON path: '$.group[6]': Last processed group[6] '5c457dfb80f3b32aec1fbefbe89a
70a63fcfae45f7f5860d65fbc32216ee1515'. com.google.gson.JsonSyntaxException:
2020-04-19T16:0n;l0:00Z"Error Code 0x1005 - Invalid Indicator Error
This error occurs when the indicator itself has an issue.
For example, the following indicator m_lena.matroschka.pl did not conform to RFC952 + RFC1123 standards since _ is not a permitted character in a valid hostname.
"errorReason":"Error (0x1005): m_lena.matroschka.pl is not valid. The indicator
'm_lena.matroschka.pl' could not be processed as a valid Host due to missing or
invalid data (summary is invalid for the given type)."Sometimes this error can be ignored. The following are examples of the system-wide exclusion list. If the ‘errorReason’ is This indicator is contained on a system-wide exclusion list, it is recommended to skip the indicator.
'Error (0x1005): https://threatconnect.com is not valid. This indicator is
contained on a system-wide exclusion list.'{'errorReason': 'Error (0x1005): https://pastebin.com is not valid. This
indicator is contained on a system-wide exclusion list.'}Error Code 0x1006 - Invalid Group Error
This error occurs when a user tries to add a group and there is a required missing field.
For example name field in the tag was incorrectly labeled incorrectly as na2me.
"Error (0x1006): Test15Adversary is not valid. A tag within Group d5ef12e8e767f37cbc
11ce662844c3cba30e341d0809ad4889f16ae7f088bb63 had a missing or empty 'name' field." Error Code 0x1008 - Indicator Limit Error
This error occurs when the limit of indicators is reached on a source.
For example, the batch module is submitting indicators belonging to a certain source/owner. However, the number of indicators allowed for that owner is 0.
'Error (0x1008): Import Failure New indicators in this batch[14] would exceed the
number of allowed indicators [0] for this owner'Error Code 0x100A - Duplicate Item Error
This error generally happens where there is a duplicate indicator in the same batch process with the caveat that the indicator does not already exist in the system.
The following code example shows how that might occur with a new indicator. Note that this will not produce a duplicate item error if the indicator already exists in the system. In this instance, the following would actually merge the attribute ‘Description’ with the ‘First Scene’ and ‘Last Scene’ attributes.
self.batch = self.tcex.batch(owner=self.args.tc_owner)
indicator = self.batch.address('123.132.1.11',
rating='2',
confidence='45')
indicator.attribute('First Seen','2020-11-09T19:26:18Z' )
indicator.attribute('Last Seen','2020-11-10T19:26:18Z' )
indicator2 = self.batch.address('123.132.1.11',
rating='2',
confidence='45')
indicator2.attribute("Description",'Hello world')
# submit batch job
batch_status = self.batch.submit_all()Error (0x100a): Found duplicate indicator in batch job file Error Code 0x2001 - Indicator Partial Loss Error
This error generally presents as a warning and the indicator will be skipped in the batch processing due to invalid or malformed data inside the indicator JSON.
The following example shows how a malformed timestamp causes an indicator to have an invalid data. A missing Z at the end of the date timestamp '2020-11-09T19:26:18' causes a 0x2001 error.
{'errorReason': "Warning (0x2001): disksys.com.br contains invalid data that was skipped.
An attribute within 'disksys.com.br' has the following problem: Value '2020-11-09T19:26:18'
does not match the expected form at for 'First Seen'. (Expected to match regex '(?=.{20}$)(?:[
1-9]\\d{3}-(?:(?:0[1-9]|1[0-2])-(?:0[1-9]|1\\d|2[0-8])|(?:0[13-9]|1[0-2])-(?:29|30)|(?:0[13578]|1
[0 2])-31)|(?:[1-9]\\d(?:0[48]|[2468][048]|[13579][26])|(?:[2468][048]|[13579][26])00)-02-29)T
(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\dZ')."}Error Code 0x2002 - Group Partial Loss Error
This error generally occurs when there is a issue with a group JSON.
For example, sda was used for the type field in the attribute section instead of “source”.
'errors': [{'errorReason': "Warning (0x2002): Test15Adversary contains invalid data
that was skipped. An attribute within 'd5ef12e8e767f37cbc11ce662844c3cba30e341d0809a
d4889f16ae7f088bb63' has the following problem: The attribute type 'sda' does not
exist in this system."