This is a database* of Creative AI tools for those interested in incorporating machine learning (ML) and other forms of artificial intelligence (AI) into their practice. They cover a broad spectrum of possibilities presented by the current advances in ML like enabling users to generate images from their own data, create interactive artworks, draft texts or recognise objects. Most of the tools require some coding skills, however, we’ve noted ones that don’t. Beginners are encouraged to turn to RunwayML.
The database is an initiative of the Creative AI Lab (a collaboration between Serpentine's R&D Platform and the Department of Digital Humanities at King's College London). It has been customised for Stages to show only tools. For the further resources like publications, essays, courses and interviews visit the full database here. The Lab commissioned Luba Elliott to aggregate the tools listed here in 2020. To submit further tools, get in touch with the Lab.
YOLO uses machine vision to detect objects in an image. By dividing an image into 'regions' it uses single-shot algorithms to identify multiple objects of interest using a convolutional neural network.
A latent diffusion model which produces images based on text prompts, with publicly available code that can be run locally.
Chat Generative Pre-trained Transformer (ChatGPT) is a chatbot built on top of OpenAI's GPT-3 family of large language models, and is fine-tuned with both supervised and reinforcement learning techniques.
Developed by Holly Herndon in collaboration with Never Before Heard Sounds. A Voice Model which transforms uploaded audio clips into outputs recreated in likeness of Holly Herndon's voice. The Holly+ Model is a neural network trained by the artist, the first of its kind. It acts as a proof-of-concept for the need for artists to produce their own models using 'high fidelity vocal training data' which can compete with low quality models trained using public domain recordings, and of which they have initial ownership and creative control. Ownership of the model will be distributed through the Holly+ DAO.
deepdream.c is an artistic experiment trying to implement Convolutional Neural Network inference and back-propagation using a minimal subset of C89 language and standard library features.
Sema lets you compose and perform music in real time using simple live coding languages. It enables you to customise these languages, create new ones, and infuse your code with bespoke neural networks, which you can build and train using interactive workflows and small data sets. All of this with the convenience of a web-based environment.
DALL.E is a 12-billion parameter version of GPT-3 trained to generate images from text descriptions using a dataset of text-image pairs.
VFRAME is an open-source project that develops customized object detection models, visual search engine tools, and synthetic image datasets for training deep convolutional neural networks.
Jukebox is neural network that generates music, including rudimentary singing, as raw audio in a variety of genres and artist styles.
Charr-rnn-tensorflow is a character-level language model in Python using Tensorflow. It is free and requires at least intermediate coding skills.
Magenta Studio is a collection of plugins for music generation with MIDI Files. It includes 5 tools: Continue, Groove, Generate, Drumify, and Interpolate. It is free and available for Windows, MacOS and as an Ableton plugin. It does not require coding skills.
Realistic-Neural-Talking-Head-Models is able to generate a moving face based a single image. This is a free implementation and requires advanced coding skills.
iMotions integrates various sensor technologies to track different aspects of human responses to stimuli in many kinds of environments. Pricing on request.
This tool uses GPT-2 to generate scripts. It is modeled on a database of film scripts from IMSDB (The Internet Movie Script Database).