Discover more from pnote

Playful exploration of fun problems related to human and computer intelligence 💫. Featuring stretched metaphors, cyberpunk illustrations and an occasional segway to Steinbeck's post-rodeo hangover.

AI-based festival advice (how I ended up at Penetrator)

Overengineering my visit at the theater festival in Avignon

Przemek

Jul 23, 2023

Last week we talked about Off Avignon, the biggest performing arts festival in the world. With no fewer than 1500 different plays in the program, choosing between shows was overwhelming.

🤖 A grid representing many different theater plays (Midjourney)

As an experiment, we tried explaining theater preferences to ChatGPT / Bard. The idea is that we tell the bot what we like, and then ask for advice on a particular show. The AI tools are happy to comply and even estimate the probability that we will like the show, based on our stated preferences and the description of the play.

However (and this is where we stopped last week), this only works for one show at a time. We don’t have the time or patience to copy and paste 1500 show descriptions to ChatGPT. We’re on vacation after all :).

Building a robot advisor

To review all shows automatically, I wrote two small programs:

The crawler. This one had one job: go through the entire website of the Off Avignon festival and record the description of each show in a file. Even with this type of automation it was taking a few seconds to process each page. This added up to a total running time of about one hour. I went to catch a quick show while the crawler was running ⚙️.

Example show descriptions, with the part that the crawler was recording highlighted in red

The ranker. This program went through the recorded descriptions one by one. For each description, it used the OpenAI API to ask GPT 3.5 to estimate the probability that I’ll like the show, using a short prompt describing my preferences:

I'm at a theatre festival and there are a lot of shows. I'd like to see:

 - elements of improv and audience participation
 - characters in their 30s searching for meaning in life
 - dystopian commentaries on society and technology

I'd like to avoid:

 - mass-appeal comedy
 - shows intended for children or seniors
  
Based on these preferences, estimate whether I will like the show described by the following pitch in triple backticks below. Respond with as a probability number from 0% (no chance I will like it) to 100% (certain that I will like it) along with a short rationale (single sentence).

Finally, the results were sorted by rating and stored.

The results and “Penetrator”

The top plays selected by the ranker were (full results here):

Of these four, Penetrator seemed the most intriguing, and that’s the one I went to see.

It turned out to be a Scottish play of the in-yer-face genre (the most famous representative of which is Trainspotting). Two flatmates receive a surprise visit from an old friend who had spent the last few years in the military. Now he’s back, hiding from a mysterious “Penetrator” bent on hunting him down.

It may be a placebo effect, but I loved it :).

The next frontier for AI assistants: bulk jobs?

What have we learned from this exercise?

That 1500 shows is a lot of shows. That models like GPT can understand a description of our preferences and evaluate a show description against them. That full end-to-end automation like crawling a website and evaluating each page of the program against our preferences is out of reach of the chatbot assistants for now.

For now… because as we’ve seen, the missing glue part of crawling the website and making GPT queries is not hard to automate using a few small programs. AI assistants capable of handling all of this may be on the horizon.

The gotcha is the cost. The ~1500 GPT queries I made to compile my ranking cost me a total of 1 USD. If we had to work with a 100x larger data set, that would be $100. So any AI system capable of doing this kind of bulk work automatically would likely need to come up with a way to inform the user of the cost of computation and charge accordingly.

Postcard from Avignon

This post completes our first two-part series ! We’ll move on to different topics the next time :). Meanwhile, have a great week 💫

– Przemek

pnote