Machine Hallucinations is an ongoing project of data aesthetics based on collective visual memories of space, nature, and urban experiences. Since 2016, our studio has used DCGAN, PCGAN, and StyleGAN to train machine intelligence in processing these vast datasets and unfolding unrecognized layers of our external realities. We collect data from digital archives and social media platforms and process them with machine learning classification models such as CNNs, Variational Autoencoders, and deep ranking, filtering out noise and irrelevant data points. The sorted image datasets are then clustered into thematic categories to better understand the semantic context of the data universe.

To capture these hallucinations from a multi-dimensional space, we use StyleGAN2, an NVIDIA DGX Station, 500 TFLOPS of AI power, and the world’s fastest workstation for leading-edge AI research and development. StyleGAN2 generates a model for the machine to process the archive and the model is trained on subsets of the sorted images, creating embeddings in 4096 dimensions. To understand this complex spatial structure visually, we use dimensional-reduction algorithms, such as cuml-UMAP, projecting to a navigable 3-dimensional universe. This projection enables the audience to use an interactive browser to virtually fly around this latent space and record their own journeys.

The final stage of production reveals Refik Anadol Studio’s pioneering work in machine-generated aesthetics and synesthetic perception through various forms of data paintings, sculptures, and latent cinematic experiences. Fluid dynamics has inspired Anadol since the inception of Machine Hallucinations. The studio’s exploration of digital pigmentation and light through fluid-solver algorithms accelerated by GPU computation and real time ray-traced lighting made possible by NVIDIA RTX GPUs brings this inspiration to life in a masterfully curated, multi-channel, self-generating experience.

Space Dreams transforms a vast data set of 1.2 million images captured from the International Space Station, along with satellite images of Earth's topology, into a dynamic data painting. Each version of the machine’s dream sequences is derived from a different generative adversarial network (GAN), exploring AIs capacity to reach its own subconscious and offering an avant-garde form of cartographic aesthetics.

In Urban Dreams, Anadol offers new insights into the representational possibilities emerging from the intersection of advanced technology, urban memory, and contemporary art. The work uses over 100 million images from New York City and Berlin, focusing specifically on typical public spaces. The content is used to train a StyleGAN2 to identify and learn patterns hidden in urban scenery, showcasing how the elusive process of memory retrieval transforms into data collections.

Nature Dreams is a series of synesthetic reality experiments based on StyleGAN2 algorithms, using over 69 million images of National Parks, Iceland, and other natural wonders to train a generative model for the machine to dream about the most mesmerizing features of our Mother Nature. This transformation of the data collection becomes not just a means of visualizing information, but also a transmutation of our desire for experiencing nature into a poetic visual.

Developing the "full scene" AI-augmented artwork started with capturing 45 photos from a single location, which were then stitched (Autopano Pro) and blended (Aurora HDR Pro) into a giant (482+ megapixel) panorama and cropped and sweetened in Photoshop. It was processed with a proprietary version of Google DeepDream running on NVIDIA Quadro RTX GPUs.

Finally, Daniel experimented with tools and techniques (including effects available in Filter Forge) to further explore the graphic possibilities inherent in his “Dreamscapes.” His most recent attempts have led to a series he calls Infinite Dreams, an exploration of Cubism-inspired "refracted" dreaming.

In this video, Dreamscapes: The Tech Behind the Art, engineers Joseph Smarr (Google) and Chris Lamb (NVIDIA) offer a layman's explanation on how they modified Google's open-source DeepDream software to operate on multi-hundred megapixel images. See how this innovative process enabled computational photography artist Daniel Ambrosi to take his work to new heights.

The artists used 3D models of hundreds of insects, literature about their life cycles, and close-ups of their textures to create this unique work. The lack of large, open datasets led to a fair deal of data augmentation.

The insects were generated using a GAN trained on data from 3D models. They were then textured with convolutional neural networks (CNNs) and given names and descriptions from a fine-tuned GPT-2 language model. The process included NVIDIA Quadro and GeForce RTX GPUs with TensorFlow and PyTorch frameworks, and cables.gl for the web experience.

Artificial intelligence is used in all parts of the process—from generating 3D forms to giving them surface texture and finally, descriptions.

Scott’s ‘Figures’ dataset comprises 30,000+ unique photographs that he shot in the studio from a diverse set of volunteers over a two-year period. The usability of a neural network is often directly related to the quality of the training inputs, so carefully curating these was critical to building AI tools in his artistic practice. 

A selection of time lapse videos from his drawing sessions is translated by the “Figures” network—an NVIDIA pix2pixHD, image-to-image translation generative adversarial (GAN) network. The network continually assesses the lines, shapes, and contours of each drawing for body patterns it ‘recognizes,’ then shades and renders them as appropriately as possible.  

The drawing process for the master image for Fall of the Damned is underway. The final artwork is 2.2 meters tall, so the preparatory drawing had to be incredibly detailed. The final drawing was too big to fit in GPU memory, so was processed through the neural network in chunks of 8192x4096 before being combined into its final size of 20500x15200 pixels.

Using her own drawings, sketches, and photographs as datasets (between 100 and 2,000 samples), Helena trains her models to generate new visuals as the basis of her compositions. She uses SNGAN and CycleGAN exclusively, finding that SOTA performs poorly on small, diverse datasets. Helena uses a BYO local server, which now includes both NVIDIA Quadro and GeForce GPUs.

Having abandoned the easel in favor of GANs, Helena often organizes her works into a grid that’s randomly generated during the inference time as part of her artistic pipeline. This technique adds a layer of complexity and depth to her work.

Helena uses this proven, repeatable process to deal with GANs and showcase many images as one work. This approach has helped to define her unique design aesthetic.

The Leonardeschi were a group of artists that worked in the studio of or under the influence Leonardo da Vinci, and 225 of their paintings were used to train the NVIDIA pix2pixHD model.

The Madonna of the Carnation diagram is one example of an application of the model in which the model can map an edgemap back to a painting.

Training the pix2pixHD model with a coarse-grained labeling semantic helps to guide the model in recognizing attributes such as hair and skin.

Oxia Palus manually co-registered (aligned) the x-ray of da Vinci's Virgin on the Rocks, from Imperial College London, with the x-ray trace produced by the National Gallery.

They then further manually edited the co-registered image, adding in missing trace components and labeling attributes such as skin, hair, and clothing.

Translucent colored labeling was used to prevent saturation of the underlying co-registered image.