Creative AI Lab [tools database]
[About the Lab]

This is a database* of Creative AI tools for those interested in incorporating machine learning (ML) and other forms of artificial intelligence (AI) into their practice. They cover a broad spectrum of possibilities presented by the current advances in ML like enabling users to generate images from their own data, create interactive artworks, draft texts or recognise objects. Most of the tools require some coding skills, however, we’ve noted ones that don’t. Beginners are encouraged to turn to RunwayML.

The database is an initiative of the Creative AI Lab (a collaboration between Serpentine's R&D Platform and the Department of Digital Humanities at King's College London). It has been customised for Stages to show only tools. For the further resources like publications, essays, courses and interviews visit the full database here. The Lab commissioned Luba Elliott to aggregate the tools listed here in 2020. To submit further tools, get in touch with the Lab.

 

Tool
Object Detection, Recognition and Classification
YOLO ↗︎

YOLO uses machine vision to detect objects in an image. By dividing an image into 'regions' it uses single-shot algorithms to identify multiple objects of interest using a convolutional neural network.

ADDED
January 2023
DEVELOPER
V7
Tool
Static image
Stable Diffusion 2.0 ↗︎

A latent diffusion model which produces images based on text prompts, with publicly available code that can be run locally.

ADDED
November 2022
Stability AI
DEVELOPER
Stability AI
LICENSE
CreativeML Open RAIL-M
Tool
ChatGPT ↗︎

Chat Generative Pre-trained Transformer (ChatGPT) is a chatbot built on top of OpenAI's GPT-3 family of large language models, and is fine-tuned with both supervised and reinforcement learning techniques.

ADDED
November 2022
OpenAI
LICENSE
GNU General Public License v2.0
Tool
Audio
Holly+ ↗︎

Developed by Holly Herndon in collaboration with Never Before Heard Sounds. A Voice Model which transforms uploaded audio clips into outputs recreated in likeness of Holly Herndon's voice. The Holly+ Model is a neural network trained by the artist, the first of its kind. It acts as a proof-of-concept for the need for artists to produce their own models using 'high fidelity vocal training data' which can compete with low quality models trained using public domain recordings, and of which they have initial ownership and creative control. Ownership of the model will be distributed through the Holly+ DAO.

ADDED
July 2021
DEVELOPER
Holly Herndon
LICENSE
Holly+ DAO
Tool
Drawing
deepdream.c ↗︎

deepdream.c is an artistic experiment trying to implement Convolutional Neural Network inference and back-propagation using a minimal subset of C89 language and standard library features.

ADDED
May 2021
DEVELOPER
Alexander Mordvintsev
Tool
Audio, Music
Sema, A Playground for Live Coding Music and AI ↗︎

Sema lets you compose and perform music in real time using simple live coding languages. It enables you to customise these languages, create new ones, and infuse your code with bespoke neural networks, which you can build and train using interactive workflows and small data sets. All of this with the convenience of a web-based environment.

ADDED
May 2021
DEVELOPER
University of Sussex EMuTe Lab: Francisco Bernardo, Chris Kiefer, Thor Magnusson
LICENSE
MIT License Copyright 2020
Tool
Static Image
DALL.E ↗︎

DALL.E is a 12-billion parameter version of GPT-3 trained to generate images from text descriptions using a dataset of text-image pairs.

ADDED
February 2021
DEVELOPER
OpenAI
LICENSE
Modified MIT License
Tool
Multi-category
RunwayML ↗︎
RunwayML contains a variety of models designed for generating images, text and beyond. It does not require coding skills and costs 0.05 USD per minute for running models on a remote cloud GPU infrastructure.
ADDED
July 2020
DEVELOPER
Runway AI
LICENSE
Terms permit you to use the RunwayML Services for your personal, non-commercial use only
Tool
Static image
VFRAME: Computer Vision Tools For Human Rights Researchers ↗︎

VFRAME is an open-source project that develops customized object detection models, visual search engine tools, and synthetic image datasets for training deep convolutional neural networks.

ADDED
July 2020
DEVELOPER
Adam Harvey
LICENSE
MIT License Copyright 2018 Adam Harvey
Tool
Audio
Tensorflow WaveNet ↗︎
Tensorflow WaveNet can be used to generate audio and speech. It is free, works in Tensorflow and requires intermediate coding skills.
ADDED
July 2020
DEVELOPER
DeepMind
LICENSE
The MIT License (MIT) Copyright 2016 Igor Babuschkin
Tool
Visualisation
Image t-SNE ↗︎
Image t-SNE is a tool for visualising high-dimensional data. It is free and requires intermediate coding skills.
ADDED
July 2020
DEVELOPER
Ml4a
LICENSE
GNU General Public License v2.0
Tool
Dance and Movement
PoseNet ↗︎
PoseNet is used to estimate either a single pose or multiple poses (in multiple people) in image/video. It is a free Tensorflow implementation and requires intermediate coding skills.
ADDED
July 2020
DEVELOPER
Google
Tool
Facial Detection, Recognition and Manipulation
ofxFaceTracker2 ↗︎
ofxFaceTracker2 can be used for face detection, face landmark detection and for head pose estimation. It is an addon for OpenFrameworks, free and requires at least intermediate coding skills.
ADDED
July 2020
DEVELOPER
Jonas Jongejan
Tool
Dance and Movement
OpenPose ↗︎
OpenPose is a real-time multi-person system to jointly detect human body, hand, facial, and foot keypoints on a single image
ADDED
July 2020
DEVELOPER
CMU Perceptual Computing Lab
LICENSE
Updated Caffe version license 1.0.0rc5 to 1.0.0
Tool
Handwriting and Digits
write-rnn-tensorflow ↗︎
write-rnn-tensorflow can be used for random handwriting generation. It is a free Tensorflow implementation that requires intermediate coding skills.
ADDED
July 2020
DEVELOPER
David Ha
Tool
Music
Jukebox ↗︎

Jukebox is neural network that generates music, including rudimentary singing, as raw audio in a variety of genres and artist styles.

ADDED
July 2020
DEVELOPER
OpenAI
LICENSE
Modified MIT License Software Copyright 2020 OpenAI
Tool
Audio
Lyrebird ↗︎
Lyrebird lets you create a voice doublethat sounds like you based on an audio sample. There is also the option to overdub, which replaces recorded phrases with synthesized speech that is tonally blended in with the surrounding audio. Lyrebird is currently in private beta, but is open to hearing about potential interesting use cases.
ADDED
July 2020
DEVELOPER
Lyrebird (now owned by Descript)
LICENSE
License Grant. Subject to their Terms, Descript grants you a limited, nonexclusive, nontransferable license to access and use Descript
Tool
Handwriting and Digits
Handwriting Synthesis ↗︎
Handwriting Synthesis can be used for generating handwriting. It is a free Tensorflow implementation that requires intermediate coding skills.
ADDED
June 2020
DEVELOPER
Sean Vasquez
Tool
Audio
AudioClassifier ↗︎
AudioClassifier can be used to distinguish different sounds from each other, providing the results to other applications over OSC.It is an openFrameworks application, part of the ml4a-ofx collection. It is free and requires basic coding skills.
ADDED
June 2020
DEVELOPER
Ml4a
LICENSE
GNU General Public License v2.0
Tool
Audio
Audio t-SNE ↗︎
Audio t-SNE places similar-sounding audio clips near each other and plays them as you hover the mouse over each clip. It is free and requires basic coding skills.
ADDED
June 2020
DEVELOPER
Ml4a
LICENSE
GNU General Public License v2.0
Tool
Audio
Audio Style Transfer ↗︎
Audio Style Transfer synthesizes audio by independently manipulating style and content. It is free and available as Tensorflow, Torch and Lasagne implementations. It requires basic coding skills.
ADDED
June 2020
DEVELOPER
Dmitry Ulyanov and Vadim Lebedev
Tool
Music
Nsynth ↗︎
Nsynth can be used to generate sound at the level of individual samples. It provides artists with intuitive control over timbre and dynamics and the ability to explore new sounds. It was trained on a dataset of musical notes. It is free and requires intermediate coding skills.
ADDED
June 2020
DEVELOPER
Google and DeepMind
Tool
Music
CycleGAN-Music-Style-Transfer ↗︎
CycleGAN-Music-Style-Transfer realises music style transfer between different musical domains (classic, jazz,pop). It is a free Tensorflow implementation and requires intermediate coding skills.
ADDED
June 2020
DEVELOPER
ETH Zurich
LICENSE
MIT License Copyright 2018 Sumu Zhao
Tool
Handwriting and Digits
Convolutional VAE ↗︎
This Convolutional VAE tool demonstrates how to generate images of handwritten digits by training a Variational Autoencoder. It is free, runs as a Colab notebook and requires intermediate coding skills.
ADDED
June 2020
DEVELOPER
Google
Tool
Drawing
sketchRNN demo ↗︎
This is an interactive web experiment that lets you draw together with a recurrent neural network (sketch-rnn). It is free, web-based and requires no coding skills.
ADDED
June 2020
DEVELOPER
Google
LICENSE
Attribution 4.0 International (CC BY 4.0)
Tool
Handwriting and Digits
Four Experiments in Handwriting with a Neural Network ↗︎
This is a series of web-based experiments in handwriting with a neural network such as generating new strokes based on your handwriting input. It is free and requires no coding skills.
ADDED
June 2020
DEVELOPER
Google
Tool
Handwriting and Digits
Digit Fantasies by a Deep Generative Model ↗︎
This is a web-based demo that generates digits. It is free and requires no coding skills.
ADDED
June 2020
DEVELOPER
Durk Kingma
Tool
Handwriting and Digits
RNN Handwriting Generation Demo ↗︎
This is a web-based demo of generating handwriting using RNN. It is free and requires no coding skills.
ADDED
June 2020
DEVELOPER
Alex Graves
Tool
Music
MuseNet ↗︎
MuseNet can generate 4-minute musical compositions with 10 different instruments, and can combine styles from country to Mozart to the Beatles. The dataset comes from a variety of sources including ClassicalArchives, BitMidi and MAESTRO. It is free and the web-based version does not require coding skills.
ADDED
May 2020
DEVELOPER
OpenAI
LICENSE
For use of outputs created by MuseNet, please cite this blog post as
Tool
Text
textgenrnn ↗︎
Textgenrnn allows you to easily train your own text-generating neural network of any size and complexity on any text dataset with a few lines of code, or quickly train on a text using a pretrained modeI. It is is a Python 3 module on top of Keras/TensorFlow. It is free and requires at
ADDED
May 2020
DEVELOPER
Max Woolf
LICENSE
MIT License Copyright 2017-2020 Max Woolf
Tool
Text
Char-rnn-tensorflow ↗︎

Charr-rnn-tensorflow is a character-level language model in Python using Tensorflow. It is free and requires at least intermediate coding skills.

ADDED
May 2020
DEVELOPER
Sherjil Ozair
LICENSE
The MIT License (MIT) Copyright (c) 2015 Sherjil Ozair
Tool
Text
GPT-2 ↗︎
GPT-2 is a transformer-based language model. It was trained on WebText, a dataset consisting of the text contents of 45 million links posted by users of the ‘Reddit’ social network. It is free and requires at least intermediate coding skills.
ADDED
May 2020
DEVELOPER
OpenAI
LICENSE
Modified MIT License Software Copyright 2019 OpenAI
Tool
Dance and Movement
Everybody Dance Now ↗︎
Everybody Dance Now enables motion transfer. Givena source video of a person dancing,it can transfer that performance to a novel target. It is free and requires advanced coding skills.
ADDED
May 2020
DEVELOPER
UC Berkeley
Tool
Music
Folk-rnn ↗︎
Folk-rnn enables you to generate folk music. It is free. There is a web-based version and also a code repository.
ADDED
May 2020
DEVELOPER
Bob Sturm
LICENSE
The MIT License (MIT) Copyright (c) 2015 Bob L. Sturm, Jõao Felipe Santos, Iryna Korshunova
Tool
Drawing
DoodleClassifier ↗︎
DoodleClassifier enables you to train a classifier to accurately recognise drawings from a camera. It is an openFrameworks application, part of the ml4a-ofx collection. It is free and requires intermediate coding skills.
ADDED
May 2020
DEVELOPER
Ml4a
LICENSE
GNU General Public License v2.0
Tool
Drawing
sketchRNN ↗︎
Sketch-rnn can produce sketches of common objects. It has been originally trained on the QuickDraw dataset, which contains rough human-drawn images representing hundreds of classes. It is free and requires at least intermediate coding skills.
ADDED
May 2020
DEVELOPER
Google
LICENSE
Attribution 4.0 International (CC BY 4.0)
Tool
Music
Magenta Studio ↗︎

Magenta Studio is a collection of plugins for music generation with MIDI Files. It includes 5 tools: Continue, Groove, Generate, Drumify, and Interpolate. It is free and available for Windows, MacOS and as an Ableton plugin. It does not require coding skills.

ADDED
May 2020
DEVELOPER
Google
LICENSE
Copyright 2016 The Magenta Team. All rights reserved.
Tool
Gesture recognition
Gesture Recognition Toolkit ↗︎
Gesture Recognition Toolkit is designed for real-time gesture recognition. It is a free, cross-platform, open-source, C++ machine learning library. It requires advanced coding skills.
ADDED
May 2020
DEVELOPER
Nick Gillian
Tool
Dance and Movement
Beyond Imitation ↗︎
Beyond Imitation is a set of open-source tools for the research of experimental dance. It is free and has examples that you can play with in browser and some where coding skills are required.
ADDED
May 2020
DEVELOPER
Yale University
Tool
Drawing
Autodraw ↗︎
AutoDraw converts your sketches into professional drawings. It is free, web-based and requires no coding skills.
ADDED
April 2020
DEVELOPER
Google
LICENSE
Attribution 4.0 International (CC BY 4.0)
Tool
Facial Detection, Recognition and Manipulation
FaceSwap ↗︎
FaceSwap recognises and swaps faces in pictures and video. It is a free, powered by Tensorflow, Keras and Python and will run on Windows, macOS and Linux. It requires intermediate coding skills.
ADDED
April 2020
DEVELOPER
deepfakes
LICENSE
GNU General Public License Version 3, 29 June 2007
Tool
Static Image
Deep Dream Generator ↗︎
DeepDreamGenerator can apply the DeepDream aesthetic to an image or apply another artistic style to it. It is a free website and does not require coding skills.
ADDED
March 2020
DEVELOPER
Small team of enthusiasts
LICENSE
CC 3.0 This means that you can use it commerically as long as you attribute the technology
Tool
Static Image
Deep Dream ↗︎
DeepDream can be used to change the aesthetic of an image, creating a multicoloured hallucinogenic appearance by using a neural network to find and enhance patterns in images. It was originally trained on the ImageNet dataset. It is free, available in Colab and requires intermediate coding skills.
ADDED
March 2020
DEVELOPER
Google
LICENSE
CC 3.0 This means that you can use it commercially as long as you attribute the technology
Tool
Static Image
Neural-style ↗︎
Neural-style combines the content of one image with the style of another. It is free and requires intermediate coding skills.
ADDED
March 2020
DEVELOPER
Justin Johnson
LICENSE
MIT License
Tool
Multi-category
Ml5.js ↗︎
Ml5.js is a set of machine learning tools for the web. The library provides access to machine learning algorithms and models in the browser, building on top of TensorFlow.js with no other external dependencies. It is free and requires basic coding skills.
ADDED
March 2020
DEVELOPER
Ml5.js
LICENSE
MIT License
Tool
Multi-category
Wekinator v2.1.0.4 ↗︎
The Wekinator allows users to build new interactive systems by demonstrating human actions and computer responses. It is free and does not require coding skills.
ADDED
March 2020
DEVELOPER
Rebecca Fiebrink
LICENSE
GNU General Public License
Tool
Static Image
DeepArt ↗︎
DeepArt changes an image into the style of an artist. It is a free website and does not require coding skills.
ADDED
March 2020
DEVELOPER
Bethge Lab
LICENSE
MIT License
Tool
Static Image
GauGAN ↗︎
GauGAN, named after post-Impressionist painter Paul Gauguin, creates photorealistic images from segmentation maps. It is free, web-based and does not require coding skills.
ADDED
March 2020
DEVELOPER
NVIDIA
LICENSE
Licensed under public domain
Tool
Multi-category
Ml4a-ofx ↗︎
Ml4a-ofx is a collection of real-time interactive applications and associated scripts for working with machine learning. It is free, requires openFrameworks and intermediate coding skills.
ADDED
March 2020
DEVELOPER
Ml4a
LICENSE
GNU General Public License v2.0
Tool
Multi-category
Ml4a ↗︎
Ml4a is a set of guides and resources designed to enable artists to work with a variety of machine learning techniques. It is free and requires a willingness to learn some coding skills.
ADDED
March 2020
DEVELOPER
Ml4a
LICENSE
GNU General Public License v2.0
Tool
Static Image
BigGAN ↗︎
BigGAN generates high quality photorealistic images. It was originally trained on the ImageNet dataset. It is a free PyTorch implementation and requires at least intermediate coding skills.
ADDED
March 2020
DEVELOPER
Andrew Brock/DeepMind
LICENSE
MIT License Copyright 2019 Andy Brock
Tool
Static Image
SPADE ↗︎
SPADE creates photorealistic images from segmentation maps. There are several free implementations available including in PyTorch and Tensorflow. It requires at least intermediate coding skills.
ADDED
March 2020
DEVELOPER
NVIDIA
LICENSE
Attribution-NonCommercial-ShareAlike 4.0 International
Tool
Static Image
StyleGAN2 ↗︎
StyleGAN2 is a state of the art image generation model used to create photorealistic images with minimal technological artifacts. This is a Tensorflow implementation and requires advanced coding skills.
ADDED
March 2020
DEVELOPER
NVIDIA
LICENSE
Copyright (c) 2019, NVIDIA Corporation. All rights reserved. Nvidia Source Code License-NC
Tool
Static Image
Artbreeder ↗︎
Artbreeder is a tool for collaborative creation based on mixing images together. It is based on the BigGAN model. It is a free website and does not require coding skills.
ADDED
March 2020
DEVELOPER
Joel Simon
LICENSE
CC0 license. This releases any image you create on Artbreeder into the public domain.
Tool
Static Image
DCGAN ↗︎
DCGAN is an image generation model. It is a free Tensorflow implementation and requires intermediate coding skills.
ADDED
March 2020
DEVELOPER
OpenAI /indico
LICENSE
MIT License
Tool
Moving image
Vid2Vid ↗︎
Vid2vid enables high-resolution photorealistic video-to-video translation. It can be used for turning semantic label maps into photo-realistic videos, synthesizing people talking from edge maps, or generating human motions from poses. This is a free PyTorch implementation that requires intermediate coding skills.
ADDED
March 2020
DEVELOPER
Nvidia
LICENSE
Copyright (C) 2017 NVIDIA Corporation. Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu. All rights reserved. Licensed under the CC BY-NC-SA 4.0 license
Tool
Static Image
Pix2pix simple demo ↗︎
This is a simple implementation of the pix2pix paper on the browser using TensorFlow.js. It is free and does not require any coding skills.
ADDED
March 2020
DEVELOPER
Zaid Alyafeai
Tool
Moving image
Few-shot vid2vid ↗︎
Few shot vid2vid photorealistic video-to-video translation. It can be used for generating human motions from poses, synthesizing people talking from edge maps, or turning semantic label maps into photo-realistic videos. This is a free PyTorch implementation that requires intermediate coding skills.
ADDED
March 2020
DEVELOPER
Nvidia
LICENSE
Nvidia Source Code License (1-Way Commercial) – NVIDIA CONFIDENTIAL
Tool
Moving image
Realistic-Neural-Talking-Head-Models ↗︎

Realistic-Neural-Talking-Head-Models is able to generate a moving face based a single image. This is a free implementation and requires advanced coding skills. 

ADDED
March 2020
DEVELOPER
Vincent Thévenin
LICENSE
GNU General Public License
Tool
Static Image
Pix2pix ↗︎
Pix2pix is used for image translation e.g. synthesizing photos from label maps, reconstructing objects from edge maps, and colorizing images.There are free implementations in Torch, Tensorflow and others, most requiring at least basic coding skills.
ADDED
March 2020
DEVELOPER
UC Berkeley
LICENSE
Copyright (c) 2016, Phillip Isola and Jun-Yan Zhu All rights reserved
Tool
Static Image
CycleGAN ↗︎
CycleGAN is used for image-to-image translation. It can generate photos from paintings, turn horses into zebras, perform style transfer and more. It is free and there is a variety of PyTorch, Tensorflow and other implementations available, most requiring at least basic coding skills.
ADDED
March 2020
DEVELOPER
UC Berkeley
LICENSE
Copyright (c) 2017, Jun-Yan Zhu and Taesung Park All rights reserved
Tool
Text
Torch-rnn ↗︎
Torch-rnn is a character-level language model in Python using Torch. It is free and requires at least basic coding skills
ADDED
March 2020
DEVELOPER
Justin Johnson
LICENSE
The MIT License (MIT) Copyright (c) 2016 Justin Johnson
Tool
Emotional Recognition
iMotions ↗︎

iMotions integrates various sensor technologies to track different aspects of human responses to stimuli in many kinds of environments. Pricing on request.

ADDED
November 2019
DEVELOPER
iMotions
Tool
Gesture recognition
ofGVF ↗︎
ofGVF enables realtime gesture recognition and variation tracking. It is free, available for MasOX or as an openFrameworks add-on and requires intermediate coding skills.
ADDED
June 2019
DEVELOPER
Baptiste Caramiaux
LICENSE
GNU Lesser General Public License Version 3, 29 June 2007
Tool
Facial Detection, Recognition and Manipulation
OpenFace ↗︎
OpenFace is a facial behavior analysis toolkit including facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation. It is free, available for Windows, Ubuntu and Mac and requires at least intermediate coding skills.
ADDED
May 2019
DEVELOPER
CMU MultiComp Lab
LICENSE
Copyright 2017, Carnegie Mellon University and University of Cambridge
Tool
Object Detection, Recognition and Classification
ConvnetClassifier ↗︎
ConvnetClassifier enables you to train a classifier to recognize any objects, persons and other visuals you show it via your webcam. It is an openFrameworks application, part of the ml4a-ofx collection. It is free and requires intermediate coding skills.
ADDED
May 2019
DEVELOPER
Ml4a
LICENSE
GNU General Public License v2.0
Tool
Object Detection, Recognition and Classification
YOLO: Real-time-object-detection ↗︎
YOLO is a system of detecting objects in real-time. It is free, requires Darknet and intermediate coding skills.
ADDED
March 2019
DEVELOPER
Joseph Redmon
Tool
Object Detection, Recognition and Classification
Teachable Machine ↗︎
Teachable Machine is a web-based tool that lets you train a computer to recognize your own images, sounds, & poses. It is free and does not require coding skills.
ADDED
February 2019
DEVELOPER
Google
Tool
Object Detection, Recognition and Classification
Image Classifier ↗︎
This tool classifies an image using a pre-trained model. The model was trained on ImageNet. It is free, web-based and requires basic coding skills.
ADDED
February 2019
DEVELOPER
Ml5.js
LICENSE
MIT License
Tool
Emotional Recognition
FaceReader ↗︎
FaceReader recognises specific properties in facial images, including the six basic or universal expressions as well as a 'neutral' state and 'contempt'. Pricing on request.
ADDED
February 2019
DEVELOPER
Noldus
Tool
Object Detection, Recognition and Classification
DenseCap ↗︎
DenseCap detects objects in images and describes them in natural language. It is a free Torch implementation and requires intermediate coding skills.
ADDED
November 2018
DEVELOPER
Stanford
Tool
Object Detection, Recognition and Classification
Mask R-CNN ↗︎
Mask R-CNN can separate different objects in an image or a video. It gives you the object bounding boxes, classes and masks. It is a free Tensorflow implementation and requires intermediate coding skills.
ADDED
June 2017
DEVELOPER
Matterport
LICENSE
The MIT License (MIT) Copyright 2017 Matterport, Inc.
Tool
Facial Detection, Recognition and Manipulation
DeepFaceLab ↗︎
DeepFaceLab allows you to create deepfakes by replacing the face and changing political speech. It is free and there are releases for Windows, Linux and Google Colab. It requires at least intermediate coding skills.
DEVELOPER
iperov
LICENSE
GNU General Public License Version 3, 29 June 2007
Tool
Text
script_buddy_v2 ↗︎

This tool uses GPT-2 to generate scripts. It is modeled on a database of film scripts from IMSDB (The Internet Movie Script Database).

DEVELOPER
Charles Pierse
LICENSE
MIT License

SIGN UP FOR UPDATES

Creative AI lab is a collaboration between
Copyright Serpentine 2023    Privacy Policy
Creative AI lab is supported by