Tell us about your work.
I am Senior Research Staff at the Barcelona Supercomputing Center (BSC-CNS) and Principal Investigator of three projects in our group. One of these - Saint George on a Bike - is a CEF Telecom (Public Open Data) project we are working on with Europeana Foundation. It departs from our more typical focus on Smart Cities to explore how technology, and artificial intelligence in particular, can improve our knowledge of cultural heritage and make it more accessible for non-specialists. Our approach relies on neural networks, language models, and semantic inference to automatically generate enriched metadata and captions for iconographic paintings from the 12th to 18th centuries.
What was your career path to your role?
My research career has covered several different areas, connected through the common topic of programming language design. My background allowed me to apply my knowledge of hardware (automatic system synthesis during my PhD and PostDoc at MIT) to widely distributed systems, actor models, and software engineering while working at IBM Research, and embedded systems during my time as a Visiting Professor at Universidad Carlos III of Madrid. I also applied Machine Learning techniques to learn expressive models for opera singers in collaboration with Universitat Pompeu Fabra, and jointly built a simulator for infectious diseases, which we have now tuned for COVID-19 in collaboration with Carlos III University and the Carlos III Health Institute in Madrid.
What are you working on right now?
We are halfway through Saint George on a Bike, and at the point where our focus is turning to connecting the results of the data mining and deep learning approach. We have been developing this with top-down semantic inferencing to generate visual relationships and improve the labels of the objects we are detecting in paintings. It's an exciting time that opens increasingly interesting research questions!
As a group, we find cultural heritage an extremely rich domain with huge potential for artificial intelligence. While this is a field that all of us can relate to, none of us is a specialist and joint work with cultural heritage experts is fundamental. Our collaboration with Europeana has so far been very positive in this respect. This project is an open door to a field we had not considered before, and we are very excited about the opportunities and possible collaborations with different types of stakeholders for this work - from museums to people with disabilities or education.
What do you think is the biggest opportunity AI presents for the cultural heritage sector?
Data mining is typically used when vast amounts of data are available and we want to extract knowledge from it. This is not the case with cultural heritage, which has a limited number of data points (artefacts), of many different styles. This creates both a challenge and an opportunity for mixed approaches that work together to refine each other or detect inconsistencies. This is the approach we are trying right now in Saint George on a Bike, with promising results in terms of generating rich metadata about the objects, topics, and actions depicted in paintings.
I believe the greatest opportunity that this rich metadata can bring to the cultural heritage sector lies in its impact on consumers of culture, reviving and expanding the interest in culture, making it more digestible, relatable, and pervasive. This can start with school education and extend to areas such as the creative industries or cultural tourism. At a more profound level, culture reflects the identity of people and societies across time and having a better understanding of our past, its biases and inequalities, could help us become a more inclusive and tolerant society.
What is the biggest challenge?
I believe that technically the most difficult aspect for AI in cultural heritage is capturing meaning, symbols, and common sense. These are fundamentally human endeavors and constitute harder problems than simply learning correlations as they model reasoning based on knowledge that is rarely part of the input data.
The line of work we are following in Saint George on a Bike tackles the interesting technical problem of endowing AI with cultural insight. The main challenge is to understand the context at the time that an image was created, the context outside which the symbols, traditions, and rules that it reflects lose meaning.
A study from 2018 suggests that just 12% of machine learning researchers are women. What do you think can be done to encourage more women into the field?
My background is in Computer Systems rather than AI and when I was in grad school, most women in our building were in fact in AI and the numbers were reasonably high. Nevertheless, the percentage you mention refers to machine learning, which is but one field in AI. It´s true that if we look at the sheer number of papers, it may seem like it has taken over the whole field!
Computer Science has always been male-dominated; I think fields where the majority of workers are male raise problems not just for women, but for anyone who is not super-competitive, (appears) aggressive, assertive, or simply loud enough. That being said, I think affirmative action must be applied with great care. Not everyone learns or works best following the same processes, nor do they necessarily consider the same thing as a strength or a weakness. I believe we need not just give more opportunities to people that don't fit the canons, but rather allow them to do things differently and be evaluated differently for what they can bring to the table.
In my experience in academia and industry I repeatedly saw that most - not all - women are highly motivated by how their work will be applied, and tend to join areas that have some connection to the society or the individual. In AI these may be recognition (voice, face), language, cognitive processes, reasoning, or applications for education, health, or culture. I don't have a recipe for how to attract women to the field, but I think AI is better equipped than other Computer Science areas to do that. I suppose this is not the most optimistic view, given the numbers.
