Moving to new Europeana Data Model

Europeana and partners have been working hard for more than two years to change the data model behind Europeana from its original and simple ESE (Europeana Semantic Elements) to the richer EDM (Europeana Data Model). EDM allows us to make explicit links between Europeana objects, e.g. the different volumes of an encyclopedia, or between an object and contextual information, e.g. an object and the geographic location it relates to, and allows us to display more than one image for each entry.

The process of moving to EDM has been a long one, but now we are delighted to announce that EDM has been implemented both front-of-house on the Europeana portal and behind-the-scenes in the process for incorporating new datasets. For data providers, this means that they can now send us their richer EDM metadata and we can make it visible on the Europeana portal. This is a big step forward - until now we have only been able to publish data in the flatter ESE format. As we ingest more EDM data, we will also be making changes to our services, so that users can take advantage of the benefits this richer data provides.

The United Ingestion Manager - a new tool for EDM publication

To make all of this happen, we have introduced a new workflow for the ingestion of EDM metadata and a new tool - the UIM (United Ingestion Manager). With the help of several plug-ins, the UIM facilitates the different steps of the ingestion process, e.g. harvesting, mapping, de-referencing, enrichment, thumbnail caching and link checking, all of which are necessary before metadata can be published on the Europeana portal.

The first EDM publication using UIM happened in June. This first publication was only possible after extended developments and rounds of testing, which meant that the normal monthly ingestion cycles were slightly delayed. The good news is that a larger EDM publication is currently being prepared by the aggregation team, and monthly ingestion cycles will be reinstated as soon as possible.

The introduction of the UIM, with a workflow which is more automated and connected than before, and the possibility to publish EDM, will now allow the aggregation team to focus more on the quality of the metadata received and on issues like persistent identifiers, thumbnails and geo-references.

The first publication with EDM included native EDM metadata (data that we had already received in EDM format) from partners like HOPE - Heritage of the People's Europe, CARARE and MIMO - Musical Instrument Museums Online. Forthcoming publications will feature even more native EDM data from these partners as well as others such ECLAP - e-library for Performing Arts, DCA - Digitising Contemporary Art, and Hispana.

Midwinter sunset, Stonehenge from CARARE, Archaeology Data Service & Clive Ruggles, CC BY-SA

Updated Documentation

As you can imagine, during the testing and implementation process, we have learnt a lot and made further amendments to EDM, taking for example into account recommendations of the EuropeanaTech Task Force on Hierarchical Objects. To support data providers, and to reflect the changes in Europeana's data, we've updated our technical documentation on ingestion:

• EDM Definition and Mapping Guidelines - UPDATED
• FAQs - UPDATED
• Technical Requirements webpages - UPDATED
• ESE documentation - UPDATED

We also maintain a list of EDM case studies that providers willing to send us EDM metadata can use as examples. Although partners can still send us ESE for the time being, the output of the UIM workflow is EDM and providers are encouraged to now provide Europeana with EDM.

Your feedback

We are well aware that in spite of all our efforts, understanding and applying EDM is not easy. This is why we are very eager to hear any comments that could make our documentation better. We also remain deeply committed to answering questions from our partners, especially when providers are preparing metadata for ingestion! So please don't hesitate to email content@europeana.eu