Shining a Light on PGConf.EU 2024 Talk Selection

We just published the PGConf.EU 2024 schedule and I couldn't be more excited to share it with everyone. We hope you love the talks we've chosen as much as we do.

A lot of the work of the talk selection committee (a.k.a. program committee, CfP committee or CfPC) happens behind closed doors, and people are understandably interested in how many submissions we had, how the process works, how the talks were chosen etc.

This post brings some of the inner workings of the CfPC out into the open and talks about how we chose 51 talks out of a whopping 380 submissions!

Note that this blog post contains my own personal thoughts and ideas and doesn't necessarily represent the opinions of the rest of the PGConf.EU organisation team or selection committee.

Gathering the Team

I was lucky enough to have plenty of volunteers to be on the committee, so I could choose a team of 5 people (plus me) with diverse backgrounds, experience levels, roles, interests and ideas. That part is really important to make sure we create a program that appeals to all of the different PostgreSQL users that we love to see at PGConf.EU.

If you spot any of the team out and about, please say a huge thank you. The hard work, enthusiasm and professionalism of this amazing group of individuals meant it was a pleasure to work with them. I would absolutely, without hesitation work with any of them again:

Number of Submissions

We limited the number of submissions per person this year, mainly in an attempt to keep the CfPC workload reasonable, but also to improve the overall quality of the submissions.

The limit was designed to:

• Encourage speakers to think carefully about what they most want to share.
• Encourage speakers to tailor submissions to be relevant to the event.
• Reduce the disproportionate time spent reviewing the 10-20 talks each that are submitted by just a handful of people.

Since the vast majority of speakers submit just 1 or 2 talks, we set the limit to 3 to make sure it didn't impact too many people.

And thank goodness we set a limit! Even with a maximum of 3 talks per person, we had a record number of talk submissions - almost 380.

You can see how this compares to the previous 2 years:

The submissions came from over 250 individual speakers - another record!

Choosing the Talks

To get from this:

• almost 380 talks
• over 250 speakers
• representing 118 organisations

To this:

• 51 talks
• 55 speakers
• representing 26 organisations

Each member of the CfPC spent at least 30-35 hours reviewing, rating, discussing and selecting talks.

The conference website has information about the practicalities of how talks are rated and selected. I won't repeat all of that here, but will give a very quick overview:

Each committee member rates the talks from 1 (lowest) to 9 (highest), without being able to see the other committee members' ratings. We abstain from rating any submission where we cannot offer an unbiased opinion (including our own and colleagues' submissions). We then calculate the average ratings to give us a starting point to discuss which talks we might want to select.

Some More Numbers

• Average rating of submitted talks 5.8
• Highest rated talk: 8.8
• Average rating of accepted talks 7.7
• 13.4% talk acceptance rate
• 24.4% speaker acceptance rate

Talk Selection Guidelines

The CfPC needs to know what's expected of them:

• What kind of talks are most appropriate for this conference?
• What is the expected audience?
• How should talks be rated?
• What are the conference organisers looking for in the overall program?
• What are the goals and vision of the conference?
• Are there rules and guidelines to keep in mind?
• How many talks are needed?

I asked the PGConf.EU organisation team to help me put together a set of talk selection guidelines for the CfPC to answer these and other questions. In summary the guidelines said that:

• We want a balanced program of talks, with a focus on PostgreSQL.
• Talks should be reviewed consistently, fairly and thoroughly.

A Balanced Program

A “balanced program of talks, with a focus on PostgreSQL” is defined as a program including talks that are:

• On a range of topics.
• Suitable for different experience/expertise levels.
• Aimed at various types of PostgreSQL user
• From both new and established speakers.
• Given by a diverse group of speakers representing as many people as possible in the PostgreSQL community.
• Not dominated by any single organisation or speaker
  (we actually try to limit to one talk per speaker).
• Of interest to more than a very small audience.
• Not commercial talks, "sales pitches" etc.
  (these would only be appropriate for the separate "sponsor track").

Fair and Thorough Review

Reviewing talks "consistently, fairly and thoroughly" implies, amongst other things, that committee members should:

• Spend a sufficient amount of time reviewing each submission.
• Abstain on rating any submission where they cannot offer an unbiased opinion.

I also asked the committee members to include a short comment for each submission. It creates a bit of extra work, but it's really useful:

• You have to think about, and explain to yourself and others, why you're rating a submission a particular way.
• It reminds you why you voted a particular way when you look back later.
• It's useful during discussions about which talks to accept, especially if multiple talks have a similar rating.
• It makes it easy to respond to any speakers who ask for feedback about why a talk wasn't selected.

Feedback from the Committee

Note that we are ALWAYS happy to provide feedback.
There can be lots of reasons for a talk not being selected:

• The abstract may not have been clear.
• The title may not have grabbed the committee's attention.
• There may have been 10 other submissions on the same topic.
• The committee may have loved it but had to say no
  because there were too many other great submissions.

Having that information can allow you to make changes to improve your chances of being accepted next time.

A Note on Diversity

We said that we wanted talks "given by a diverse group of speakers representing as many people as possible in the PostgreSQL community".

By diversity, we don't just mean things like gender, ethnic origin, disability, LGBTQIA+ etc. but also things like new vs. established speakers, professional background, PostgreSQL experience, end-user vs. Postgres hacker and more. There are many reasons why this is important, and there are lots of people who speak much better than I do on the topic, so I won't try to list the reasons here.

Our main approach to this was to talk about and share the CfP as widely as we could both within and outside of the Postgres community, and to personally reach out to as many people as possible to encourage them to submit talks.

It's extremely difficult for us to measure how well (or not) we're doing in terms of diversity, because we do not currently collect any diversity-related information from speakers. We're considering the best way to do this effectively and sensitively for future events, taking inspiration from other communities where this is already commonplace.

PostgreSQL Europe has also launched a Diversity Task Force initiative that will discuss this and other questions. Work will be starting very soon, with an official launch workshop during PGConf.EU.

Stand by for more information and a call for volunteers!

Speaker "Benefits"

Accepted speakers are eligible to register for the conference using the free of charge “speaker” registration type, and are usually invited to a pre-conference speaker dinner. Speakers (or, more often, a speaker's employer) usually fund their own travel and accommodation. If this isn't possible, we ask speakers to let us know during the submission process so that we can try to allocate budget to provide some assistance.

Looking Forward

PGConf.EU, just like PostgreSQL itself, is getting more and more popular each year, and some of our processes are having to change to reflect that. Just like all organisations, there are some things we do really well, and some things we could do better.

We've tried some new things this year (for example asking for volunteers for the CfPC and limiting the number of submissions per speaker) and there are plenty of other things that we could think about for future events, including:

• Put out a public call for volunteers for the CfPC to make sure that everyone in the community has the chance to participate.
• share the CfP more widely so that it reaches even more potential speakers.
• Split the CfPC into smaller teams and have each of those teams review a subset of the submissions.

News from Other Talk Selection Committees

If you're interested in how talks were selected for some of the other events related to Postgres this year, here are some posts/documents that you might find interesting:

• About Talk Selection for POSETTE: An Event for Postgres 2024 (Claire Giordano)
• Building the PgConf.Dev Programme (Paul Ramsey)
• pgDay Paris 2024 Program committee feedback