How to configure a basic TAXII server?

STIX/TAXII is the most common method of sharing Threat Intel in the world of security operations today. However, paying up for costly threat intel subscriptions might not always be feasible which would have brought you to the idea of building your own TAXII server. By doing this, you can automate the process of manually adding IOCs from various sources to your SIEM tool.

This article would be discussing the following:

1. How to get your TAXII server up and running?
2. How to configure the server?
3. How to convert threat intel data to STIX format?
4. Pushing data to your server
5. Testing your server
6. Polling through a SIEM tool

In this article, however, I won’t be discussing how to aggregate threat intel data — if you are making a TAXII server you must already have a pool of data aggregated & analyzed using openly available threat intel sources or other sources that you use to feed data manually to your SIEM tool. You can check an example here — this simple python script fetches and parses domain related data using AlienVault’s API and saves it to a remote SQL DB.

1. How to get your TAXII server up and running?

If you are well versed with TAXII specifications, you would understand that building these many configurations from scratch is not a viable option. You can instead use a python library that provides you a great base to start with — -the OpenTAXII Library.

It implements all TAXII services according to TAXII specification (version 1.0 and 1.1). On top of these services, it also delivers additional functionality such as customizable APIs + Auth!

Steps to set up:

1. Create a virtual environment(Suggested for protecting system-wide python):
(Windows)
python3 -m virtualenv venv
/venv/Scripts/activate.bat
(Linux)
virtualenv venv
. venv/bin/activate

2. pip install opentaxii
3. pip install gunicorn

Now before jumping on to step 4- there are certain changes you need to make in order to avoid STIX compatibility issues at a later stage. Go to the folder inside your environment where opentaxii is installed — it can mostly be found on this path: venv/lib/site-packages/opentaxii.

- In _version.py: change __version__ = ‘0.2.1alpha’ to __version__ = ‘0.2.0’
- In management.py: update return jsonify(token=token.decode(‘utf-8’))
to return jsonify(token=token)
- Now navigate to opentaxii/auth/sqldb/api.py: update payload = jwt.decode(token, self.secret) to payload = jwt.decode(token, self.secret, algorithms=[“HS256”])

4. Run the server with default configuration using gunicorn:
gunicorn opentaxii.http:app — bind 192.168.1.27:9000 — config python:opentaxii.http

To see server logs run this instead:
- gunicorn opentaxii.http:app — bind 192.168.1.27:9000 — config python:opentaxii.http >> /var/log/opentaxii.log &
-(separate terminal) tail -f /var/log/opentaxii.log

2. How to configure the server?

This step is considerably easy. There are two key yaml files that need your attention:

1. defaults.yml — Add “secret: YOUR-SERCRET-KEY” in the parameters section of auth api. Incase you’re NOT running on localhost and have hosted your TAXII server elsewhere, you might want to update the domain to “your.domain:80” to avoid errors during polling.

2. data-configuration.yml — this is a working configuration file. It is recommended that you keep “accept_all_content” and “authentication_required” values unchanged, they can cause certain errors while polling. Apart from that you can add/remove the following block under “collections” to configure.

Also check out the last section of the yaml — it allows you to create users and define permissions for them.

Once you are through updating these two yaml files, run the following commands within your virtual environment. (Please do verify that the path to these files is correct)

export OPENTAXII_CONFIG=/venv/lib/python3.7/site-packages/opentaxii/sample.yml

opentaxii-sync-data /venv/lib/python3.7/site-packages/opentaxii/data-configuration.yml

You can now kill and rerun gunicorn using the same command (gunicorn opentaxii.http:app — bind 192.168.1.27:9000 — config python:opentaxii.http)

3. How to convert threat intel data to STIX format?

The taxii server usually does not a restriction on the format of the data that you push — JSON/XML etc. However, when using sophisticated threat intel tools — you can use python-stix library — it is an easy way to create STIX XMLs if you have handy raw threat intel data. This is a good example to start with.

4. Pushing data to your server

If you’re done configuring and running the server and have certain XML files ready- all you need now is a small script that helps you to push data to your server but before that:

pip install cabby

Why do we need cabby? To push data to a taxii server, a taxii client is needed, cabby handles the same for us in the following code:

One thing you will need to change is the collection name and the location to the xml file you created! Run the following file after making those changes:

from cabby import create_client
client = create_client('localhost',
                        use_https = False,
                        port = '1234',
                        discovery_path='/services/discovery-a')
client.set_auth(
    username='admin',
    password='admin',
    # URL used to obtain JWT
    jwt_auth_url='/management/auth'
)
# Check the available services to make sure inbox service is there
services = client.discover_services()
print(f"Services: {services}")
# Get the data that we want to send
with open("examples/stix/stuxnet.stix.xml") as stix_file:
    stix_data = stix_file.read()
binding = 'urn:stix.mitre.org:xml:1.1.1'
# URI is the path to the inbox service we want to use in the taxii server
client.push(stix_data, binding,
            collection_names=['collection-a'],
            uri='/services/inbox-a')
print(f"Successfully exported to TAXII server.")

5. Testing your server

Let’s first check if Gunicorn is running:

Now, we can check various database tables of SQLite3 to check if data has been pushed, although not required I recommend you to still do this — you would get a better idea of the DB schema and server as a whole.

The image shows the various tables present. Two important things to note here are inbox_messages and collection_to_content_block.

Let’s check the schema for inbox_messages:

This is the main table where your data will be pushed. If you would like to see the contents (the XML you pushed in the previous step) try using:

Select * from inbox_messages Where id==1;

collection_to_content_block is th eother important table — it maps the blocks of data pushed to the various collections you have defined.

Now, let’s move on to the commands provided in the official documentation and see the ideal responses that you should get:

A TAXII Service is a single unit of capability within TAXII. TAXII defines four TAXII Services:

• Inbox Service — Used by a TAXII Client to push information to a TAXII Server. This is what we already used in the previous section.
• Poll Service — Used by a TAXII Client to request information from a TAXII Server.

taxii-poll — path http://192.168.1.27:9000/services/taxii-data — username admin — password admin -c “Hybrid Analysis”

The above requests data from the “Hybrid Analysis” collection and as you can see in the image above it polls all available blocks (5, in this case)

• Collection Management Service — Used by a TAXII Client to request information about available Data Collections or request a subscription.

taxii-collections — path http://192.168.1.27:9000/services/collections_a — username admin — password admin

This requests for a list of all collections from ‘collection management a’ of the taxii server. As you can see in the image above shows all information about every collection present.

• Discovery Service — Used by a TAXII Client to discover available TAXII Services — this is the key service — any SIEM tool first uses this service to get a list of collections and services.

taxii-discovery — path http://192.168.1.27:9000/services/discovery — username admin — password admin

6. Polling through SIEM tools

I tested this server using IBM QRadar. After adding the threat intel extension, and creating a service — you can head on to Admin -> Stix/Taxii Configuration -> Add Threat Intel Feed.

Now, enter <your-domain>/services/discovery and the username and password that you defined (preferably for admin) in the data configuration yaml.

On the next window, if everything is configured correctly — you should see the collection list — you can now configure your polling interval and reference sets(Hashes/IPs/etc.)

This is how the signatures look after polling is completed. You can also check out admin -> reference sets to see the data that has been polled.

Thank you for giving this a read! :D

To help me improve this article or just start a conversation — you can hit me up on LinkedIn.

Cheers!