S2E2: Randomized Benchmarking with Dr Ellen Derbyshire

How can a completely random process be used to test the accuracy of quantum computers?

This week we're featuring Dr Ellen Derbyshire, a postdoctoral researcher at the Dahlem Centre for Complex Quantum Systems, Freie Universität Berlin. Dr Derbyshire obtained her PhD from the University of Edinburgh before taking up her current postdoctoral position.

Photo credit: Ezekial (Ezy) Galan (http://www.ezekialgalan.com/photography).

Transcript

🟢 Steven Thomson (00:06): Hi there, and welcome to insideQuantum, the podcast telling the human stories behind the latest developments in quantum technologies. I’m Dr. Steven Thomson, and as usual, I’ll be your host for this episode.

(00:17): In previous episodes, we’ve talked about what quantum computers might be able to do for us in the near future and the challenges in developing them. We’ve talked about various types of quantum computing hardware, and how error correction methods are being developed in order to make current generation noisy quantum computers a bit more practical, but we haven’t so far talked about one key issue, something that we have to ask before we start trying to correct errors or interpret the results that we get. How do you know that your quantum computer is doing what you think it’s doing? This week’s guest is working on solving exactly that problem. It’s a great pleasure to be joined today by Dr. Ellen Derbyshire, a postdoctoral researcher at Freie Universität Berlin. Hi Ellen, thank you so much for joining us here today!

🟣 Ellen Derbyshire (00:58): Thank you for having me. I’m very happy to be here.

🟢 Steven Thomson (01:01): So before we get into the details of how we benchmark quantum systems, let’s first talk a little bit about your journey to this point, and let’s start right at the beginning. What first got you interested in quantum physics?

🟣 Ellen Derbyshire (01:13): Well, this is a good question. I think that I always enjoyed learning about quantum mechanics at high school over other physics subjects, but I think what really got me interested in quantum physics was when I was at university, and funnily enough, I found quantum theory classes…I found them to make more sense than say, statistical classical mechanics and all the classical physics, solid state physics and all of that. I found quantum mechanics and advanced quantum theory…I was just ready to accept these strange concepts of quantum entanglement and quantum superposition, and it felt somehow more graspable to me than the macro world that we live in, which is an odd answer, I suppose. But that’s what got me into it. It felt like something that I naturally had an affinity for and I found it the most fascinating subject I suppose at university.

🟢 Steven Thomson (02:24): I think you’re definitely the first guest who said that quantum physics made more sense to you than classical physics.

🟣 Ellen Derbyshire (02:30): Yeah, I don’t think I can really pinpoint why I, I wouldn’t say it conceptually made more sense when I learned it. I thought, oh, yeah, okay, I can be down with that. That’s fine.

🟢 Steven Thomson (02:45): So at what point did you decide that you wanted a career working in quantum physics?

🟣 Ellen Derbyshire (02:49): So I would say that that was during the fourth year of my university degree. Before that, I had my heart set on particle physics, and I had dreams of being the next Stephen Hawking from when I was a teenager. I would say that’s what got me excited about physics - string theory, particle physics and particle theory. And then during the fourth year of my integrated Master’s degree, I had a Master’s project which was about interpreting results from the Large Hadron Collider and trying to superimpose a theory about extra dimensions on these results, which when I first read about it, I was like, this is so cool and so exciting, and it’s a very fascinating topic. But when I was actually doing the work, it just didn’t resonate with me in the way that I thought it would. And then I had classes on quantum computing and communication and quantum theory, and those were the fields that I found most exciting. So I thought, okay, this is actually the career I want. And now instead of knowing, okay, I want to be a physicist, now I know, okay, I want to be a quantum physicist. That’s what I care about.

🟢 Steven Thomson (04:18): So what do you think it was that first grabbed your attention about the sort of particle physics and the high energy physics, as opposed to quantum?

🟣 Ellen Derbyshire (04:26): Well, I mean, think there’s a lot of overlap because of course, quantum theory is very relevant and necessary for particle theory. And you need quantum field theory, for example, for all these particle particle physics fields…that shows you how much I know about particle physics now! But for what made me more excited about quantum and quantum computing specifically, was this idea that we could utilize quantum effects for moving around information and interpreting information. And it felt like a more practical application to me, which is also funny now that I’ve been in the field for longer. But it felt like a practical application of these quantum effects that I found so interesting. And it also felt more cutting edge at the time because it’s quite a new field, and it felt like extremely exciting to me. Not that other long established fields are not exciting, they are extremely exciting, but just for me, it felt like, okay, this is where I want to be.

🟢 Steven Thomson (05:43): There’s something really exciting about being around at the start of a field, I think when there are so many unknowns, even the toolkits, even figuring out what’s a sensible question to ask. Even these things sometimes aren’t very clear at the beginning of a field. And yeah, you’re right. The last, I don’t know, five years even of quantum information has been just full of really rapid explosive progress in so many different directions because it is such a new field.

🟣 Ellen Derbyshire (06:09): Yes, precisely. And I didn’t know what quantum information meant or what quantum computing was until the fourth year of my degree. So that in itself felt like a very new field to me. Perhaps other people did, but for me, it just blew my mind that you could use quantum mechanics to compute things, really.

🟢 Steven Thomson (06:35): So that’s maybe a good point to ask, how did you get to where you are now? Can you give us a quick summary of your career to date?

🟣 Ellen Derbyshire (06:42): Yes. So I would start back in high school. As I said, I had visions of being a kind of crazy physicist, super genius, but I didn’t really like physics at school until I was about 14 or 15. I loved maths, I loved humanities subjects. And I think a friend once said to me, you know, should just try and listen more in our physics classes, you might find something interesting because I would really switch off. And I think part of that was being told that physics is not exciting, it’s the least fun science subject. And when I sat down and I focused, it was interesting. It was like, oh, okay, that’s quite cool. And I started to read outside of school and read about different pop science books. And one of those books was a Brief History of Time by Stephen Hawking. I tried to read that when I was a teenager, and obviously I didn’t understand any of it, but it caught my interest and things you would see on TV and documentaries.

(07:59): I don’t know, it just made me feel like physics was a subject where you could apply maths, a subject that I really loved to all these really cool science fiction sounding topics. So at that point, I just said it, okay, what I would like to be is a physicist. And so that was the goal, and I was like, I know I have to get a degree in physics. And then during my degree I kind of realized, oh, to be a “physicist”, you need to have a PhD. So the goal always was to be a theoretical physicist. And then along the way, I realized what kind of physicist I wanted to be.

(08:42): So yeah, at university, as I said, I got into quantum computing quite late, and then it was a case of trying to find a PhD position also with funding attached because it can be expensive to do a PhD. And luckily for me, the field that I chose had this explosion. So often PhD positions are funded, and for me, I applied to a few positions and I didn’t really know a lot about what kind of topic I would want to do within this field, and I think that you learn that as you go along. So I found my first position with Edinburgh University. I had a friend who I was working on quantum information with and trying to improve my knowledge on quantum information. And she was a PhD student at my undergrad university – UCL in London – and she was a PhD student in quantum, so she knew of some people working on certain things in the field.

(09:54): So what I wanted was to work on quantum algorithms, and she suggested Elham Kashefi, who was my supervisor for my PhD. So I spoke to Elham and luckily that got the ball rolling, and she accepted me as a student. But I had applied to a few other positions before that, and I took a year off between my university and my PhD to kind of work and figure out what kind of field I wanted to work in. And then it was Elham that really got me onto this, my current expertise, I would say, which is randomized benchmarking, which I’m sure we’ll talk about later. And for my postdoc position, I felt like this group, which is with Jens Eisert, would allow me to learn so much because the people in this group have a lot of broad expertise and a lot of their expertise overlaps with the topics that I was interested in during my PhD. So that’s kind of how I got here. I think for everybody, you can only see how you got somewhere in hindsight and in general, it’s a combination of choices you make and a sprinkle of luck always as well. So yeah, that’s what I would say.

🟢 Steven Thomson (11:19): I think it’s interesting often to talk to people who are working in this field and discover that they started somewhere else. It’s actually a story that I’ve heard quite a few different times where people get fascinated by some of the perhaps slightly flashier fields or the fields that were around, I guess when perhaps we were teenagers in high school, we’d read these books on as you say particle physics, astrophysics, things like this. We go to university excited to study those and then somehow find the reality doesn’t quite match our expectations. Maybe that’s, maybe that’s because they’re older fields and all that’s left are hard questions, or maybe that’s because our expectations were just wrong because we’ve seen all these sort of glamorous TV shows and books and things, and we had too high expectations. I don’t know. But it’s very interesting to hear that quite a lot of people that I’ve spoken to have said that they initially started somewhere else and then for various reasons they ended up in quantum physics. So then, if you hadn’t gone down that road and you hadn’t ended up in quantum physics, what do you think you might have done instead?

🟣 Ellen Derbyshire (12:20): I think that’s a very good and very big question. Funnily enough, I could probably categorize my job expectations as before 15 and after 15. I would say before the age of 15, I was mostly interested in actually becoming an actor, and I enjoyed acting and writing mostly, and I thought that my life would end up somewhere in one of those two things. The idealist in me would say, I would like to be doing something as creative as acting or writing as a job. But I think that those fields are even more competitive than academia. And something else that I would be interested in and could maybe be more realistic alternative would be working in a social type job. So potentially civil service or a policy making job. I think that I would find a job like that challenging and fulfilling.

🟢 Steven Thomson (13:30): Yeah. Wow, that’s a really interesting answer. You seem to have some very different interests, though. You could have gone down some very, very different routes and be doing something completely different.

🟣 Ellen Derbyshire (13:39): Yeah, well, I don’t know. I think that there is an inherent curiosity in both wanting to act and wanting to research. It’s curiosity about people or curiosity about how the universe works. So for me, they’re not that dissimilar, even though they on the surface are very, very different.

🟢 Steven Thomson (14:02): Yeah, that’s a fair point. So let’s talk a little bit about the research that you do then. Can you give us a quick summary then of what is the field that you work in? What’s the big picture goal of your field and how does your work fit into that picture?

🟣 Ellen Derbyshire (14:16): Okay. So the field I work in, I would describe broadly as quantum information, quantum computing, and I would say that the big picture goal is to solve difficult problems with quantum mechanics. So that’s a very vague goal, but with that, I mean the goal is to create a universal quantum computer that is fault tolerant, so a quantum computer that we can use and we can trust to solve problems that we maybe can’t solve classically. My main focus is on noise characterization. So figuring out how well a quantum computer might be working, because they can be subject to a lot of environmental interference. That means that they’re not running as you would expect them to. And I think that this fits into the wider goal because we are still very far away from a universal fault tolerant quantum computer. Even with recent advances where we are expecting to have small fault tolerant quantum computers in the near future, we will need noise characterization and understanding of physical quantum systems to scale up to a large enough quantum computer such that we can solve the problems we want to solve. So I think that noise characterization and understanding how a quantum device is really working physically is definitely going to be relevant for a long time to come.

🟢 Steven Thomson (16:01): So where does this noise come from? And I mean, we have classical computers all around us. We don’t tend to think about noise influencing classical computers. Where does the noise come from in a quantum computer and why does it seem to be such a significant problem more so than with classical hardware?

🟣 Ellen Derbyshire (16:17): Right. So that is a good question. I think that firstly, we’re still at the beginning of really understanding quantum effects and quantum physics, and we’re trying to manipulate these natural systems that kind of have a mind of their own. And at the moment, the quantum computers or the quantum devices we have, they’re made up of fragile…I would say fragile objects that we interpret as quantum bits. And the environment can absolutely kind of change the physical dynamics that are going on in such a device. Usually a quantum computer is a highly isolated system and it needs to be kept at a certain temperature and basically are, I would just say more fragile than the classical computers that we have because classical computers are…their functioning is based off of macro effects that we can observe more easily and they work at room temperature and they have many, many more years of work behind them to deal with noise. But because we’re so early in the quantum computing game, I think that this noise is very relevant because we don’t even really understand every aspect of it yet.

🟢 Steven Thomson (17:55): Okay. So that’s the big picture goal of the field that you’re working in. Let’s talk a bit more about your specific work. So if I were to attempt to summarize your work in a single phrase, I might say something like randomized benchmarking. Can you break that down for us? What does that phrase mean?

🟣 Ellen Derbyshire (18:12): This is a really good question because when I first heard the term randomized benchmarking, it did not make any sense to me. The word benchmarking to me just in language terms means comparing one thing to another and randomized obviously conjures up images of random things, random objects. So the idea of benchmarking something random was extremely baffling. But I think that the wording is not totally incorrect because what we’re trying to do is benchmark a device. So we’re trying to figure out the quality of quantum gates on a specific physical device. So randomized benchmarking is kind of, it’s an experimental method at its heart that relies on strong kind of mathematical theory. But the idea behind it is to get a single measure of quality of quantum gates running on a quantum device. So a single measure of how noisy a quantum device is when a particular set of quantum gates are implemented on it.

(19:30): And the kind of special thing about randomized benchmarking is that errors from preparing an initial quantum state and errors from taking a measurement of a quantum state are kind of filtered out. So you really just get a measure of the quality of quantum gates that make up a quantum circuit on your computer. And the randomized part is important because essentially you are randomizing the quantum gates that you’re applying to your device because you are trying to get a measure of the average over a big family of gates. So you have to randomize to emulate the average over the kind of whole space of quantum gates.

🟢 Steven Thomson (20:22): So it sounds like there’s two parts to this then. So one is if I were to prepare a quantum state and some generic device and try and do some other form of benchmarking, then any error in my preparation of the state might affect the results. So you’re saying that this protocol that you’re working on it, it doesn’t take this into account, so you’re really just measuring the error in the quantum computer doing some quantum computing operation, and you’re discounting any error in actually setting up the quantum computer in the first place.

🟣 Ellen Derbyshire (20:51): Right, yes. And I mean the key thing though is that you aren’t getting a measure for how well a particular quantum computation or a specific quantum computation is running. It’s really about how well a device can run certain quantum gates. So it’s a much more general measure of the quality of your device in running certain operations, but as a whole, rather than I can run this quantum algorithm and now I know on this device it’s going to have this noise, it’s much more of a generic measurement.

🟢 Steven Thomson (21:37): I see. So it’s more like, let’s say in an ideal perfect world, maybe we’d want to run every possible calculation on the machine and then take the error of each one and then average out the errors. What you’re saying is what you’re essentially doing is just taking a random sampling of all the possible calculations you could ever dream of doing in one of these machines, and as long as you pick this random sample in an intelligent way, it tells you something about the average error across all of these possible computations that you could do. Is that about right? Is that fair?

🟣 Ellen Derbyshire (22:09): Yes and no. I would say that is, I think that’s a good way of summarizing it. I think that the thing with randomized benchmarking is it, it’s a term to cover so many different protocols now for benchmarking quantum devices that there are some that do a lot more than that.

🟢 Steven Thomson (22:27): So it sounds like your work is actually quite interdisciplinary in a lot of ways then because you’re applying these sophisticated mathematical techniques to understand the gates and so on, all this mathematical structure, but you’re talking about real devices, real world devices actually doing these things and real world devices, they’re not perfect. There are a bunch of different ways that you can try to build quantum computers and they all have slightly different behavior and different characteristics. Have you found it challenging at all to be working in this interdisciplinary way where you’ve got to be thinking about the real world, but you’re still thinking about these sort of formal mathematics and algorithms and so on? Have these two worlds come together naturally for you, or has it been challenging to keep both of these things in your head all the time?

🟣 Ellen Derbyshire (23:13): Yeah, I think that I essentially, depending on who I’m working with, I play the role of the downer because if I’m working with experimentalists or people who are more familiar with experiment, I tend to be asking a lot of questions about the theory and the opposite be said for the theorist. So I’m like, okay, but does this work? What are our real limitations? And neither side want to hear about the other. In a lot of cases, I think the gap is being bridged a little in quantum computing because the field itself is interdisciplinary, so you have to be aware of all the different facets that come together to create quantum computer. But I think for me, when I started my degree, I was in a computer science department and I’d never had any computer science background. So first aligning physics and computer science was difficult, and then later trying to understand more complex mathematics that I didn’t have experience with in my undergraduate degree and aligning that with actual physical experiments. I would say it has been a challenge, but it’s also a nice place to be because I think that you, there’s always something new to learn and you can stay grounded in a way if you’re trying to do something interesting in the theoretical space, but you’re also trying to restrict yourself to a potentially more practical, physically feasible area that kind of creates exciting solutions to problems usually.

🟢 Steven Thomson (25:10): So what does the future hold for these sorts of benchmarking techniques then? In particular, as quantum computers get…as the technology gets more mature, as they presumably get more and more accurate? Will there be a point when we no longer need to characterize noise, will noise be just no longer a problem, or do you think there’s always going to be a place for these sort of techniques to characterize how these systems behave?

🟣 Ellen Derbyshire (25:33): I think that is a big question for one person to predict, but my personal belief is that for the foreseeable future, these techniques are very relevant because of the scalability issue and being able to scale beyond accurate quantum computers of a smaller size and making them big enough and funnily enough, small enough. So large enough in terms of what they can do, but small enough in terms of being practical. Doing that requires a lot of complicated physical engineering and understanding physical quantum devices and knowing how noise affects the devices in different environments. And so for me, these noise characterization is still very fundamentally important and it will be for the foreseeable future. Is there a world where we have universal quantum computers that are fault tolerant and we don’t have to worry about noise at all and they’re just running how we expect? Hopefully. But I think that in the early days noise, understanding noise and understanding architectures is extremely useful to even get to that point.

🟢 Steven Thomson (27:03): I see. So it’s a field that will have a future for a long time to come then?

🟣 Ellen Derbyshire (27:07): I think so. And I also think with any field there are certain things that pop out of different theories and different approaches that can be applied to different fields, especially within physics and computer science. So hopefully it will have applications even beyond.

🟢 Steven Thomson (27:32): All right. A couple of final questions to wrap up with then. So there’s one question that I always ask to every guest on this podcast, and that’s to say that physics, as I’m sure everyone is aware, has for a long time been a very male dominated field, dominated mostly by white cisgender men. Over the course of your career, have you seen anything changing in terms of diversity in gender balance between the different places you’ve worked and also given that you’ve worked between physics and computer science, have you seen any differences in the diversity of your colleagues in these two different fields?

🟣 Ellen Derbyshire (28:11): I think this is a really great question to be asking. I think that personally it’s difficult to answer whether I have seen many changes in terms of diversity in my own career because I’ve been in research groups that are less diverse than others and more diverse than others or been at various conferences with the same thing. So I don’t know that there’s been a trajectory towards a more diverse inclusive space, but I would say that my conversations have certainly changed. So I know that when I was an undergraduate, I would not have felt so comfortable discussing the issue of being a minority and the issue of other minorities not being included in this physics space. Whereas now I feel that the more people I meet in, I don’t know if it’s in quantum computing or in physics or in computer science, but the more people I meet in my career, the more I’m able to have these conversations and the more that people seem invested and interested in this.

(29:35): And I don’t think that’s a universal experience, and it could be to do with the confidence that I’ve grown in trying to address equitable research groups and trying to address the inequality that we face. But I do think that there are definitely more conversations than at least I was observing in my undergraduate degree, but that could have been at the stage of my career that I was at. I think in terms of physics and computer science, my perception is that computer science is a little bit more equal in terms of gender. And I dunno if that’s correct, but that’s my perception. I think that physics seems to be more male dominated, especially at the top of the pile, shall we say. So yeah, I do think we have a very long way to go. I think that sometimes what is potentially not understood in these drives to create more diverse and equitable research groups is that that leads to more innovative research and more collaborative environments for people to work in.

(31:04): And I think that sometimes it’s easy to lose sight of that if you are working in a field, you are the majority, you think everything is growing great for you, but this environment could be better for everybody. And there have been published studies that say that this kind of interaction with lots of different people from different backgrounds does lead to more innovative solutions to problems and potentially more kind of exciting outcomes. So it’s inherently in your favor as a researcher to work in these environments. And I think that the more senior that you get, the more it’s your responsibility to address these issues and create a more inclusive environment for people. Obviously, I don’t expect there to ever be a kind of utopia and physics where every single identity is represented, but there’s definitely space for it to be a more inclusive – I’ve said it inclusive so many times – but a more inclusive environment.

(32:16): And I think that that would create a, I don’t know, it’s kind of like the one thing feeds the other. So if you have a more inclusive research group, you are going to attract more different types of people. And so we seem to be stuck in this realm of the research groups are maybe dominated by a specific majority, which makes it less attractive for various minorities. And we are feeding this, it’s like a self-fulfilling prophecy. So there needs to be a lot more action taken in trying to change this. Yeah, I could say a lot more on this if you want…!

🟢 Steven Thomson (33:01): I think it’s an interesting topic because as you say, if you want your group to do the best research possible, you want the best people possible regardless of background. And then you would think that this would naturally just result in a balance, and it absolutely doesn’t because of this entrenched bias that we’ve had for so long. And I do wonder as more businesses get involved in quantum computing and so on, whether just because of a desire to have that cutting edge, just make more profits than you can best serve, just whether that is going to be enough to incentivize people to make more diverse hiring choices. It would be interesting to see – not that I’m saying businesses are utopias either, but I’m curious to see if the field will evolve in that direction from a purely commercial motive. It’s not the best motive, but if the outcome’s a good outcome, perhaps it’s still worth it.

🟣 Ellen Derbyshire (33:59): Yeah, maybe. I mean, I do think that something that businesses have more than universities, from my limited understanding is more of a pressure to address diversity. I think that you can get away with not addressing these issues in a university. And I think a lot of people have that mindset that because you want the best people who are doing the best research, it has naturally ended up in this majority rule situation. But of course, that’s ignoring societal systemic bias and the fact that, as I said, if there’s a majority, you are less likely to want to be a part of it sometimes, even if you have the necessary skills. But I would be interested to see if, yeah, quantum startups and quantum computing if the industry has an effect on this, because I do have friends who work in the industry and it seems to be a cause of anxiety for industry, potentially more than it is for universities. So maybe it will have an effect.

🟢 Steven Thomson (35:21): Yeah, that’s an interesting question. It would be interesting to see how this evolves over the years to come.

🟣 Ellen Derbyshire (35:27): Yeah, definitely.

🟢 Steven Thomson (35:28): One final question then, if you could go back in time and give yourself just one piece of advice, what would it be?

🟣 Ellen Derbyshire (35:36): One piece of advice, oh…I think the advice I would give my younger self would be that you have a lot of goals and you have a lot of milestones that you think you want to reach. And that will be the point where you say, okay, I’m fulfilled. I’m happy, I did all I wanted to do, but that’s not really how it goes. You reach your milestone and then you create a new one for yourself and you keep going. And these goals are obviously useful in keeping you motivated in work and in life, but I think my advice would be to not measure yourself by milestones and instead try to embrace everything that you’re experiencing and learning in the moment, which is potentially a cliche, but I…I’m still trying to follow that advice myself.

🟢 Steven Thomson (36:42): I think that is a fantastic piece of advice to end with, in fact. Okay. So if our audience would like to learn a little bit more about you, is there anywhere they can find out about you on the internet, social media, anything like that?

🟣 Ellen Derbyshire (36:54): All my social media, it’s private at the moment, but I am on LinkedIn and if you just Google “Ellen Derbyshire Edinburgh”, you should find enough information about me and I would say watch this space for a website that will hopefully be coming this year.

🟢 Steven Thomson (37:14): Okay, perfect. Well, and when that website arrives, we’ll be sure and leave a link on our own website to it. In the meantime, thank you very much, Dr. Ellen Derbyshire for your time today.

🟣 Ellen Derbyshire (37:24): Thank you.

🟢 Steven Thomson (37:26): Thank you also to the Unitary Fund for supporting this podcast. If you’ve enjoyed today’s episode, please consider liking, sharing and subscribing wherever you’d like to listen to your podcast. It really helps us to get our guest stories out to as wide an audience as possible. I hope you’ll join us again for our next episode. And until then, this has been insideQuantum. I’ve been Dr. Steven Thomson, and thank you very much for listening. Goodbye.