Open Source Information by MISP, OSINT
One of the nice new features by MISP is including feeds from different open source intelligence feed providers.
How does it work? Basically the feeds are provided as a JSON feed, you can browse them within MISP, import them individually or subscribe to the feed to get automatic updates.
Using OSINT feeds within MISP
First I assume that you upgraded MISP to the latest version. This is straightforward by pulling the latest version from Github.
Once this is done, log in to your MISP instance and go to Sync actions, List Feeds. You will then get a list of the available open source feeds.
Then you will need to enable or edit a feed. This will bring you to a screen where you can set the default distribution level and most importantly, the default tag.
For my setup I use the TLP:White tag. Remember that these tags are defined in the taxonomies that you have defined in MISP (see Event Actions -> List Taxonomies). I use TLP:White because OSINT information is by default TLP:White. Not sure about the TLP code? Use the post How to use the traffic light protocol – TLP.
Do not forget to tick the checkbox Enabled because otherwise the feed will not be enabled.
Once you have edited the feed you will return to the overview of available feeds.
You can browse the feed content. Do this for the feed that you enabled.
This will result in a an overview of the open source events from that feed. You can now select an invidividual event to be included in your MISP instance.
Basically that is all there is for including an open source feed to your MISP data.
- Enable a feed
- Set the distribution-level and tag it
- Sit back and enjoy
I want to contribute open source intelligence feeds
Using the OSINT feed is great but maybe you also want to contribute your OSINT feed to the community? You can. And it’s fairly easy. For this you need to use a Python script : PyMISP.
PyMISP is a Python library using the MISP Rest API. In essence this means that instead of crafting MISP API requests you can use Python request to interact with MISP.
Installing PyMISP is described on the GitHub page.
git clone https://github.com/CIRCL/PyMISP.git cd PyMISP python setup.py install
Basically it needs an API key and a URL. This can be set in any file you desire as long as you include it in your Python script that calls PyMISP. As an example you can use this in you Python scripts
from pymisp import PyMISP from cudeso import misp_key from cudeso import misp_url from cudeso import misp_verifycert
This will import the API key and url from an external file.
Using PyMISP for OSINT
Once you have setup PyMISP you can use one of the example scripts to generate the OSINT feed. In the folder PyMISP/examples/feed-generator/ there are two files that you need to edit. Note that the OSINT generator script does not use the configuration script of PyMISP described above.
First you need to edit the file settings.py. The important settings to change are
url = '<MISP-URL>' key = '<MISP-automation-key>' outputdir = 'output-dir'
The MISP url (url) defines where the script can find you MISP instance. The automation key (key) is the key set by your automation user. Finally the outputdir sets where you want to output the files. Note that this directory will hold all the MISP events. You have to make sure that this directory
- is writable by the generate script
- is a separate directory
- accessible by the public if you want to publicize your feed (for example in your web directory).
Once you have set these settings you can run the feed generator script by issuing
This will generate all the MISP events in the output directory set by “outputdir”.
Make your feed known!
If you are happy with your feed you should announce it to CIRCL to have it included as one of the OSINT feeds in the next update of MISP. The easiest way for doing so is via the MISP GitHub repository.
Existing open source feeds
The current release of MISP contains these OSINT feeds
I maintain the feed for botvrij.eu. If you want to include a description of a threat info then you can send me a MISP XML file or a pointer to a public description.