Dataharvest 2025 - the European Investigative Journalism Conference

DESCRIPTION:

For monitoring purposes and to inform our reporting, it’s helpful to keep an eye on trends over time in some datasets, for example when tracking the progress of an mpox outbreak, monitoring pre-election polls or investigating migration trends.

This is where automation comes in handy: with the data being automatically grabbed from the source, reconfigured and channeled into a chart for visualization, allowing data journalists and their colleagues to notice report-worthy trends early on.

To establish such a workflow, we have been relying on free tools like Datawrapper and Github Actions to run a Python script.

Participants will be guided through setting up such a workflow step-by-step.

LEARNING OBJECTIVES:

🚀 In this session you will learn how to...
- collect data from a url
- parse the data into the needed format using Python's `pandas` library
- use Python's `datawrapper` library to create a chart
- set up the script to run automatically on Github Actions

🔍 PREREQUISITES & TOOLS:

- ✅ Datawrapper API token
- ✅ Github Account (if you want to automate the chart update)
- optional: Code text editor (if you prefer working with your own code for automation), such as Atom or Sublime Text
- optional: Distill browser plugin (if you want to update on click)
- optional: basic understanding of Python/coding helpful, but not required

Getting the most out of your investigative data requires more than ad-hoc scripting. Deep research requires a persistent state, data model, user tagging, collaboration, access management, and automatic updates. In short, you need a research application. In this session, you'll learn to use an open source, low-code application to turn a simple ETL into a complex app.

To follow along comfortably, you should be familiar with basic python scripting and REST APIs.

AI-models are great tools for structuring unstructured data, especially if you know some basic Python. Those language models can be used to find and classify all arguments in a bunch of documents, make statistics of political debates or hunt for greenwashing in corporate reports. In this session we will extract all relations in the Game of Thrones series and plot them as a network map. Come along if you want to know who slept with whom! (Here is an example of what it looks like: https://lasseedfast.se/got/ ). After this session, you will be able to use AI in combination with Python to systematically extract pieces of information from a large volume of data.

To follow along, participants need a basic understanding of Python.
Install Ollama  and download one of the models to follow along on your own laptop.

Learn how AI can sort through images and video to help you wrangle footage from protests and riots (NYT), analyze trends on TikTok (Washington Post), keep an eye on your local school board meetings (Hearst), measure the effects of congestion pricing (Bloomberg), and a hundred other tidbits for when the cameras might be rolling.

With a little Python and a dash of foundational knowledge, this session will cover downloading videos, building and evaluating transcripts, splitting scenes, categorizing images, and detecting/counting/tracking objects.

Participants will get the most out of this session if they have a working knowledge of Python. To follow along, you should have Jupyter installed on your computer or a Google account to use Google Colab. Additional materials and installation tips will be available at https://github.com/jsoma/dataharvest25-ai-images-video

How can can journalists use metaprompting to let AI build system instructions for assistants to empower investigations? Through examples of iterative metaprompting Rune Ytreberg will show you how to master the art of efficient prompting, and use metaprompting to build AI assistants to unlock the black box that is AI.

After attending this session you will be able to use advanced prompting techniques (meta-prompting) to use a large language model (LLM) to make system instructions. These advanced prompts are detailed instructions that customize AI assistants supporting investigations. Among others they are used in Open AI and Anthropic projects, and RAG platforms like Kotaemon or Anything LLM.

You need no special knowledge to attend this session, but if you would like to follow along on your computer you will need some basic knowledge of prompting tools such as Chat GPT, Claude or similar, and will need to have signed up for a paid version of your chosen AI platform before the session.