Paintings Based on Machine Learning Generated Images
Intrigued by the idea of merging mindfulness and computers I wanted to delve into machine learning and how a machine could potentially envision yoga practice. When training new machine learning models, it is often common to use an enormous image dataset of varying results to produce a very diverse model. This challenge often results in bias which is difficult to avoid. I decided to make my work personal to me eliminating this problem as all I am interested in are the visual outputs based on the machine’s interpretation of my physical body.
I trained a model using a dataset of images taken while I was practicing yoga. The results were generated by the machine learning model breeding images of my deformed body with soft and rounded features often missing limbs.
From these results I painted a triptych using acrylic paint inspired by Francis Bacon. The final piece is a moving painting where I project a video of the generated latent space walk onto the paintings. These videos show the morphing of one generated ‘pose’ to another resembling transitions within yoga as well as the subtle trembling of the body when holding a pose or breathing.
This is a further exploration of my Runway ML yoga poses trained model outputs. I wanted to explore the concept behind them further and to create an outcome in the physical world away from the screen.
To this point the process depended on pictures of my physical body and of my efforts and practice of yoga, these were then used as a dataset to train a machine learning model using Runway ML. This model then generated new images based on the dataset resulting in very abstract images. The obvious connections to the original dataset and images of me is firstly the colour palette of the blue mat, my black clothes, varying shades of the cream/beige/white colour of my wall changing with the light sources throughout the day/nighttime the pictures were taken, the wooden floor and the colour of my skin sometimes merged with other colours resulting in areas of grey.
These are 3 of the output images:
I then used an object detection model where the AI model looks for shapes and labels them along with a percentage of certainty. The model also highlights the shape it is seeing in a bright colour, I explored these shapes in other experiments.
These are the object detection results for the images show above:
The results were cat, cow and dog. I chose these specific images because of these results. These are actually well known poses in yoga and I felt it was interesting that the AI was naming these images it generated with no connection to yoga poses with these names.
Throughout exploring this I was looking at paintings by Francis Bacon who is an artist I researched and wrote about 4 years ago when studying Advanced Higher Art but his painting became very relevant to these results. Looking at his work I always really enjoyed his triptychs and how they were these abstract forms, figures and self-portraits.
I really wanted to create my own triptych based on the AI yoga images especially because of the connection with the names I felt the shapes would work very well alongside each other and be much more impactful together than on their own.
I’m really happy with these results and I hope to explore this further and maybe consider using a bigger canvas. I feel this is a nice outcome for the concept I have been exploring and these paintings are almost like self-portraits but on the other hand I also consider them to be a collaboration with AI.
6th April
Over spring break I wanted to work on creating a large scale triptych and implementing advice from Paul and priming the canvas boards with blocks of colour before painting the figures.
I imagine the final piece for these paintings as a moving tryptic by projecting onto them. This is why I have chosen to add a white rectangle to the paintings and the contents and more about the projection part of the paintings is outlined on my blog in ‘Part 8 – Slitscan‘.
I used acrylic paint and 3 canvas boards 16x20inch.
I started with blocking in the colour. I kept the colour scheme minimalistic only using blue, black and white.
I then wanted to paint the ‘figures’ and I used these 3 reference images.
Reference images for paintings.
I altered the images using ‘skew’ on photoshop so that they weren’t all front facing even though this slight adjustment won’t really be visible in the final since I’m not drawing in the mat or the harsh lines to portray this distance I still wanted to slightly change them.
Friday April 9th
This week I finished the paintings
When speaking with Jen she mentioned she didn’t feel like the slitscan videos I have been working on and I wanted to project onto the painting made much sense together and I agree. I have been working on my green screen latent space videos and I decided to make specific ones to project onto these paintings. I chose to create latent space videos with generated images visually similar to the ones I had painted. This allows for an exploration of the figure in the paintings as a moving figure going through transitions just like a person practicing yoga.
I made some videos of the projections on the paintings when I set it up in real life but I realised the footage was not so clear and it would be much clearer if I worked on a video documentation mockup of this final in Premier Pro. I wanted to stage the paintings in a gallery so I used a copyright free image and then worked on how to smoke and mirrors the projection. I re-did this video 4 times until I was happy adding and changing parts I felt could be better and I am really happy with the result. I feel it is nice and clear and clearly displays how I want to stage this piece of work. I will include some images from the videos I was not satisfied with, on the other hand I was glad to see the projection and the work set-up in real life so I feel that even though the footage will not be used to present my final work it was useful for me to see my work as it would be if I could show it in a gallery.
I painted a triptych using acrylic paint inspired by Francis Bacon. The final piece is a moving painting where I project a video of the generated latent space walk onto the paintings. These videos were created the same way as the Morphing Figures outcome but when choosing the images which created the generated latent space walk videos I was careful to select images which were very similar to the figures in the corresponding paintings. This achieved an outcome where the moving projection is a deeper and more extensive exploration of the still figure in the painting.
Marta Zofia Palacz 