Inside DuckDuckGo

In this episode, Greg and John (Search) discuss local search results, and how we help you find local cafes, restaurants, and business, without knowing your precise location. Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.Greg: Hello and welcome to DuckTales where we go behind the scenes at DuckDuckGo and discuss the stories, technology and people that help build privacy tools for everyone. In each episode, you’ll hear from employees about our vision, product updates, engineering or approach to AI. I’m Greg. I work on the search engine here at DuckDuckGo. With me today is John. John, would you like to introduce yourself?John: Hi, I’m John. also work here at DuckDuckGo on the search engine and I’ve been focused on working on local search for most of my time here.Greg: Awesome. Yeah, that’s kind of what we wanted to talk about today is local search. And so I’m looking forward to chatting with you a little bit about how local search works, how we do it at DuckDuckGo and get into some of the challenges that we have as a private search engine and how we solve those. Local search generally, it’s kind of maps related searches. whether you’re searching for places, businesses, restaurants, those kinds of search queries we kind of call local searches. We have a couple different versions of a map that shows up on the search results page. When you do one of those searches, either you’re searching for something kind of in a city and you put the name of that city in your search query, or if you search for something near you, have ways to provide those results. still in a privacy preserving way. So maybe as a starting point, I thought it would be good to just, if you could walk us through a little bit of the life of a local search query, how it goes from kind of something the user has submitted to us to what we decide to show on the page.John: Yeah. So we start with a, we get a user query. So somebody type something in and sends the query to us. And we try to figure out whether that would benefit from some sort of mapping information. So maybe the query contains a number and a few words and then drive or street or road. And you know, that looks like an address. So we’ll show a map of that address. Other common patterns are, you know, common. types of businesses like restaurants, hotels, cafes, and those sometimes come in with location indications. know, cafes in New York City or cafes in Paris. And sometimes they don’t, sometimes it’s just cafes or cafes near me. So we kind of have to handle both those types of situations. And the ones that are nearby are really kind of the more interesting ones. because we have to find a way to figure out where Nearby is for the user. And we have to do that in a very privacy-respecting way because that’s our whole thing. So the browser, the web browser that you’re using will send a location. But we don’t just straight up use that because we deal with lot of partnerships where we have to get some data from our partners at Apple and TripAdvisor. And we don’t want to just send them raw user locations because that is a privacy issue. So what we do is we’ll kind of add some noise to it. So if you say you’re at, you know, this exact location in a city, instead of sending that, we’ll move you a little bit. Depending how big the city is, we might move you just a few hundred meters. If you’re way out in the country though, where maybe you’re our only user for miles, you know, we might move you a few miles. And so that... can affect your results, makes them a little less accurate, but we think we do it with as little noise as possible to get you a lot of privacy benefit while still giving you results that are hopefully very accurate to the location you’re interested in.Greg: Yeah, so there are a couple of things in there that I think are interesting to talk a little bit more about. when you kind of described how we add noise to a user’s location, there’s a lot of documentation, by the way, about this, if you want to read more on our help pages. But I’m wondering how we think about the kind of trade-off between making sure that the results are still good and the way that we’re obfuscating someone’s location is sort of sufficient from a privacy standpoint. So maybe you can talk about the first part first. How do we think about the quality of the results and whether or not we’re showing the right local results for someone when they do one of those nearby queries?John: Right, yeah, we have a lot of criteria that we check against before we show results to users. We’ll get a whole bunch of options that might be relevant to the query. And if you’re searching for a cafe, maybe a restaurant is relevant, but probably a hotel is not so relevant. And so we may make sure we get the category right first. And then we also want to make sure the location is correct. And because we’re dealing with partners, they might give us some locations that aren’t as relevant just because we don’t give them as accurate of a location. And so we’ll kind of refine those on our side without sharing some of that information with a partner to say, you you gave us this location that’s 100 miles from the user and that’s maybe because we gave you the wrong location on purpose. And so we’re not going to show that to the user, but we will show these other locations that are actually nearby and relevant to the users.Greg: Got it. Yeah, and then, I mean, even, you know, sometimes when there is a location in the query, it could be one of those that is still ambiguous to us, right? I think a lot about the difference between Paris, Texas and Paris, France. Users don’t always clarify what they mean. So we have to try to figure that out based on context too, right?John: Exactly. even some locations that are very well defined. If somebody says New York City, that’s just one place that we’re going to look. But New York City is huge. just determining which part of New York City is maybe the most relevant to send to the user is another challenge that we have to figure out.Greg: Yeah, and I mean, just we’re sort of riffing on these now, but, you know, if you search someone searches for New York pizza, right, that could be they’re looking for a pizza restaurant in New York City, or they’re looking for a restaurant near them that serves a New York style pizza. So, yeah, all kinds of things that we have to sort of contend with and try to figure out to give those right results. I’m wondering if maybe we could talk a little bit more than about. the different types of ways we actually display the results once they appear on the page because we have a couple different versions of our local modules that we show depending on the type of result and the number of results we get back. So I’m wondering if you’d say a couple things about that.John: Yeah, we have many different ways to display based on what the user is searching for. As I mentioned earlier, we have a special address display if you’re just looking for a map given a location. That’s a pretty bare result. It’s mostly just a map and the location label. Some more popular landmarks, we have a special display for that where you’ll get the map and if it’s the Eiffel Tower or something like that. Then you’ll get a picture of the Eiffel Tower and a little description that we sourced from. Wikipedia and then there’s a business searches is the next big category and those are kind of divided into two as well if you’re searching for a very specific single business like, you know Joe’s cafe and Las Vegas I don’t think there is a Joe’s cafe in Las Vegas maybe there is Then then we’ll just show you that one Joe’s cafe ideally, you know, we don’t want to show you Bob’s cafe That’s also in Vegas or Joe’s cafe. That’s in Idaho or somethingGreg: There might be.John: And then there’s the sort of category or chain type businesses where if you search for Walmart or restaurants, we’ll show you a big listing of businesses and where they are distributed on the map to let you kind of click in and explore more if you want to do details about each one individually.Greg: Got it. And then that’s gonna vary too based on the category of the result, right? I think we have some kind of special display for hotels as well.John: Yeah, we’ve just kind of introduced this and I can share this real quickly because I think it’s a pretty cool feature. So just recently we’ve added some pricing indicators. So if you’re searching for a hotel, you probably want to go stay at one of these hotels. And one thing that is important about staying in a hotel is how much is it going to cost and is it available at the time I’m traveling. So we’ve got a new interface. We just released this last month where you can put in your check-in dates, check-out dates, how many people, and it’ll tell you a pricing estimate of how much it’s going to cost for those dates. And then once you click into it, you can get different offers from different providers. So you can get expedia and go through and actually have really easy access to booking one of those hotels that you’re interested in.Greg: Awesome, yeah, I think this is a really good example of something where it’s a new feature that we’ve added that captures a particular use case that we know users have. mean, that we ourselves as users of a search engine have when you’re booking a hotel, trying to make it easier for users to complete the task that they’re doing, not necessarily just kind of see the different results. I’m wondering, maybe you could talk a little bit more about some of the other challenges around those additional use cases or business data that’s important for us to show and how we’re thinking about that and how we see that evolving over time.John: Yeah, we build this feature on a few different really important partnerships with Apple, Yelp, and TripAdvisor being the main three. And we use those partnerships to give us all sorts of data about different businesses. Yelp gives us lot of information about restaurants and new service businesses, shopping, stuff like that. And then TripAdvisor gives us hotel information and information more about that. attractions, know, museums, theme parks, stuff like that. And we’re trying to push more into using this data to give the user a lot richer experience. Some ideas, you know, including menu links for restaurants, giving the ability to order food or serve tables, call us a service business, you know, if you want to plumber, you can get it like kind of a quote directly. And all those are kind of ideas that were pushing for and hope to release soon.Greg: Yeah, definitely. There’s all kinds of different use cases that people have with businesses. know even just knowing the hours that the business is open, right, is so important. And these are things that users ask about all the time. I think if I had to pinpoint one big challenge with local in general, it’s just that the coverage for the business that the user cares about in that particular moment is so important. And so we’re constantly trying to figure out ways to expand that coverage and make sure we have the right information about as many businesses as possible.John: Yeah, exactly. We’ve done a lot recently also in terms of taking user feedback because, you know, we found users get very upset if we send them to a business and we got the hours wrong and the business is actually closed or they try to call a business and it’s the wrong phone number. So we’ve taken a lot of steps to submit or solicit feedback from users on when we get it wrong and then forward that upstream to our partners so that they can correct it and we can correct it and everybody gets a better experience.Greg: Of course. Yeah. Yeah. Yeah, definitely. I want to emphasize, if you as a user send us feedback that, know, hours are wrong or, you know, an address or a phone number, you know, we take all of that in and do our best to forward that to the source of that data and also make sure we’re not showing it to other users and kind of getting better and better at correcting that over time. Cool. Well, we’ve covered a lot of ground. Anything else you think is worth sharing and talking about with respect to local search?John: nothing comes to mind straight away. I’m hoping we can keep working on the product, making it better. think we’ve got some good expansions in the future. Some more, pie in the sky ideas we have of, you know, moving to something like a flight search or something might, come into later on. and yeah, just hoping to keep making the product better and getting users what they want and giving them the right information as quickly as, as easily as possible.Greg: Yeah, definitely. All right, well, it was really fun chatting with you about this today, John. Thanks for taking the time. yeah, maybe we’ll have a chance to talk more on a future episode about other search-related things. But yeah, thank you.John: Thanks for your talking.Greg: Bye. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit insideduckduckgo.substack.com

In this episode, Beah (Chief Product Officer) and Sveta (Design) discuss the fire button, the recent redesign, and how it improves both user privacy and delight. Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.Beah: Hello, welcome to Duck Tales, where we go behind the scenes at DuckDuckGo and discuss the stories, technology, the people that help build privacy tools for everyone. In each episode, you’ll hear from employees about our vision, product updates, engineering, or our approach to AI. In this episode, we’re going to talk about the fire button. And I will be your host today. My name is Bea and I’m on the product team. And we have as our guest Sveta. Sveta, do you want to just quickly introduce yourself?Sveta: Yeah, I’m Sveta. I work on mobile apps at DuckDuckGo for now five years.Beah: Sweet. Thanks. And you’ve been, you’re a designer by trade, right?Sveta: Yeah. I’m on the design team.Beah: And you’ve been doing a lot of work to improve the fire button on mobile. And so that’s what we’ll jump into today. Okay, so first of all, let’s start with the basics. Like, what is the fire button? What does it do?Sveta: Right, so the fire button is a button that’s located quite centrally in mobile UI and it looks like a fire. And when you press it, it removes, it deletes all of your tabs and all of the browsing data and all of the history. So pretty much removes everything, all of the browsing activity and you can start from clean slate. And it also shows a really cool fire animation when you do that.Beah: That seems like a good prompt to actually show it. So let me see if I can share my screen here. Can you see that, Sveta? Okay, then hopefully our audience members can as well. Alright, so this is the fire button here. I’m just on a website, on NASA’s website. I’m going to tap the fire button. And so I have, what are these two choices here?Sveta: So the first, the primary button will delete all of the tabs, all of this data for all sites. And the second button is a choice to delete activity for only this one tab. So it will close this tab and will delete all of the site data associated with the sites visited in this one tab.Beah: Alright, I’m going to choose that one here. Delete this tab. Awesome. All right. I just, I wanted to only delete the one because I also wanted to show like if you’re using, if you’re a Duck AI user, this is a new feature. But if you’re on a particular chat, for instance, a chat about what music your dog might enjoy, you can hit the fire button and now you have this opportunity or this option to just delete this chat. So remove it from your chat history.Sveta: Nice.Beah: Boom. All right. let me get it shared on. Okay, I don’t think we’re sharing anymore. Cool, so like what is the benefit of using the fire button?Sveta: Yes, the primary benefit is privacy reasons, because as you browse the internet and you visit sites, you accumulate different site data such as cookies or different user preferences that then tracking companies can, if you accumulate a lot of it, they can use it to identify you and connect your browsing activity to you. Sell that information to other companies to show you targeted ads. And our browsers, we actually block most of the trackers and most of the cookies, but there are still some cases when we can’t do it. And then that’s a good practice to delete all of this data every now and then. And then the second reason is just still schooled and it makes you, lets you start your next session from clean slate.Beah: Yeah, I like how those two, they’re like different reasons to use the fire button, but I kind of like how they converge too. Like part of what feels good about a clean slate and not having like stuff everywhere is, I think for me, like not having your data everywhere, you know? Like it’s just clean, it’s like a sense of control and I don’t know, care, I suppose, that you can apply more easily with fire button. Like you can tell me if this is how you would characterize it. Like you can delete data in most browsers, but it’s just a lot harder, right? Like you don’t have a button front and center with just the options that you need, like one or two taps away.Sveta: Exactly. It’s usually tucked like hidden somewhere in the settings. I think it’s actually pretty cool that we managed to turn this really mundane and boring technical thing in a really satisfying experience.Beah: Yeah, and one thing I didn’t show is that you can actually pick between, I think, three different animations. Like the fire is the default animation, but there’s like this kind of nice swirly animation and all like water animation. And then you can also just turn animations off if like what delights you is speed and efficiency and not an animation. That’s cool. Cool. Tell me a little bit about like how the fire button has changed over time.Sveta: Yeah, so what the confirmation that we’ve seen just now, that’s updated design that we’ve launched recently. But for years, I think since we first introduced the fire button in our mobile apps, we just had this system, little system confirmation to confirm, delete all of your tabs and data that was very small in the bottom of the screen and it was really easy to press. And it was of course good for users who were regular fire button users because it was quick. But then there was also a big risk that you could just accidentally press it and delete all of your tabs and data without intent.Beah: Yeah, so if your intent was to blow everything away all the time, fine, great, it works great for you. But if you wanted to be a little bit more surgical about it and keep some tabs around, then it neither worked for you. It didn’t really work for you, and if you accidentally used it, you might really regret it.Sveta: Yeah, we did get some that obviously there’s if you’ve accidentally deleted all of your tabs without wanting you would write us an angry feedback, which we sometimes received you open it and you see all caps that chances are from someone who just deleted their tabs.Beah: Yep. So take note, dear reader, if you are writing angry feedback, a kind human like Sveta is reading it and taking it to heart. And so in this case, what did we do about it?Sveta: So we’ve changed a lot how the confirmation dialogue looks like. It’s bigger, that you’ve seen it. It takes a lot of space on the screen, so it’s harder to miss. We’ve on purpose moved the buttons a little bit high up. So when you press the fire button and your thumb is right above the fire button, the confirmation buttons, they are not on the same spot. And there is an animation, pictogram that also attracts attention. it’s just everything is made for you to pause, read the label, and make an informed decision. And then another big change is that we’ve added a second option because before you could have only deleted everything, all of your tabs and all of your data. And now we have a second option that you can delete just one tab and the data for that tab.Beah: Yeah, yeah, I love, by the way, like, I’ve developed, you know, I’ve been using our mobile apps for a long time, but I’ve developed this kind of workflow recently now that we have the single tab, single chat burn, so handy, we have this kind of, we’re in the process of updating behavior when you open the app after an idle period, we take you to a new tab automatically, so you can just start typing your query or your chat or whatever. I like realize there are tabs that I want to keep around, the things I want to keep around, but there are a lot of things that I just want to do a quick search, a quick whatever, and it’s just, it’s throwaway. And like now I open the app, I’m in exactly where I need to be, I don’t have to think do I want a new tab or do I want to, you know, override this tab? And then like when I’m done, you know, five seconds later, just doop, tap, burn that tab, and it’s just, I don’t know, there’s, it’s a very satisfying and efficient workflow for me for the I don’t know, 70% of things I do that are just quick interactions with the app and don’t require any kind of historical retention. So I love it, Sveta. Thank you.Sveta: I think there’s something interesting about this approach to delete activity from one tab, because in other browsers, the way they solve this problem, the task of deleting some of the data, not all of the data, they use time. So you can delete an activity for, let’s say, last 20 minutes or last hour, but then it deletes it across other tabs, all tabs. While in our browser, like you said, you just did one thing in one tab, and then we delete that one tab without affecting any other tabs.Beah: Yeah, yeah. I think it’s a great shift. Do you want to talk at all about the design process that you used with the team to get to these improvements?Sveta: Yeah, so our first hypothesis or first direction that we explored for this new mobile confirmation was the same approach that we have on desktop browsers where we have a list of options for users. Every time you press the fire button, you can choose, do you want to delete tabs or keep tabs, delete the side data or keep it. And that is a good direction because it gives most flexibility, but it also, we realized that in mobile context, just looks, it looks heavy. It took over the whole screen, the confirmation just, we were asking users to make these choices every time. We, well, in our old design where you had to only press one button. There was something nice about that privacy package. We gave users the most private option with one button pressed. So we decided to instead just add another option so you could either delete everything. With one button press or delete one tab.Beah: Yeah, yeah, I think it’s like the fire button is such a great example of a feature that has like, sort of delight elements to it that make it really satisfying to use and sort of goes above and beyond like what, you know, the basic functionality that users might describe themselves as expecting from a browser, like, but it does provide this like really useful functionality. And it, I think it’s like the process to keep both of those things true, maximize functionality that people want and keep it delightful in its simplicity and not add cognitive load to the user experience is pretty tricky and seems like something that we navigated well in this case and got to a pretty good outcome, I think. Do you have, have we gotten feedback or do you have any indicators, how do you know if it’s a good outcome?Sveta: Yeah, we do surveys for our new users and the last survey that we did, one of the questions, asked the users a specific question about the fire button and we’ve seen that the delight. I don’t remember exactly the question, but we ask if this experience is delightful. And that has increased compared to the old design. I think we will, I also see less angry feedbacks in all caps from users who deleted their tabs without wanting to. Something that was also a challenge in this project is that this balancing, making this confirmation easy enough for old users to not lose this effect of confirming quickly and deleting data with one tap, but also adding some friction to the new user so they will not accidentally delete it. Of course, we also received some feedback from the old users who thought the new confirmation was a little too complicated. But that’s the reason now think people are liking the new design.Beah: Yeah, I feel like that’s an ongoing challenge with changing our mobile apps, right? It’s like we have this pretty big band of DuckDuckGo mobile app faithful who have a particular workflow that works for them. you know, whenever you change it, there’s some risk that it irritates people by kind of just like messing with their workflow, even if it’s a net positive improvement. Yeah, I think the fire button’s probably right at the center of that challenge because people who use it really love it and it’s like probably a big part of their experience of what makes the app special.Sveta: Yeah, and we have a lot, have millions of users who use our regular fire button users, but we want more people to feel that this is something for them. I am actually, I wasn’t a regular fire button user because it was too distractive like my tabs, but now I also use it more often myself.Beah: Yeah, yeah, same. Totally. rarely used it. Rarely did I want to destroy everything, and so same. I mean, I wish I were that person that was just like clean slate every day or every use, but of course I have several tabs sitting around that are supposed to be reminders for something, so yeah, I love it. Yeah, I was gonna ask you what your favorite thing personally about the fire button is, but you just touched on something you like about it now, which is per tab distraction. Anything else you would add?Sveta: Well, when a few times when I do actually use the delete all app option and delete all of my tabs, I have to say it really feels nice. It feels like Marie Kondo, she came and cleaned all of my tabs and it’s a good feeling.Beah: Yeah, so sometimes you look at your tabs and you ask yourself, do they bring me joy? And if the answer is no, burn them all down. Yeah, nice. So anything that you’re looking forward to, like any plans to continue improving the fire button in the future?Sveta: Yeah, I think there’s lots of things we could explore and improve. We’ve maybe trying to see if we can find a new flow or simplify the confirmation for the regular fire button users. We’re also trying out something interesting on Android now where we’ve just going to launch to release this update. switching the buttons around and we’ll have burning, deleting the one top as a primary button. See if this would make this fire button feel more, less scary, less destructive and hopefully will make more people feel like that’s something for them. That will be interesting to see how that works out.Beah: Nice. Awesome. Anything that we didn’t get to that you want to add, Sveta?Sveta: I think, so I don’t think a lot of people know, but on our iOS app, you can actually customize that fire button. If you tried it and you’re sure that you want to use it regularly or not as much, you can long press and swap it for something else. You can also go to settings, appearance and do it from there. And if you do hide the fire button from the main UI, you’ll still have it in the main view. think that’s good to share. also, yeah, just to everyone, try the fire button. Give us feedback. We read it. It will help us. Yeah.Beah: Yeah, nice. No need to use all caps, but we do read and appreciate the feedback. Yeah, awesome. Yeah, and if you were like a, if you discounted the fire button in the past because it was too destructive, let this be your sign to give it another try. Now that we’ve made these improvements to it, I think you might really like it. Cool. Well, thanks so much for joining, Sveta. I really appreciate it. And thanks to everyone for watching.Sveta: can try with one tap, yeah, small steps. Thank you, was fun.Beah: and see you later. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit insideduckduckgo.substack.com

In this episode, Greg (Product) and Karl (Design Engineering) discuss how we build, design and decide when to show Instant Answers. Plus, how we’re balancing instant answers with optional AI generated answers. Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.Greg: Hello and welcome to DuckTales where we go behind the scenes at DuckDuckGo and discuss the stories, technology and people that help build privacy tools for everyone. In each episode you hear about our vision, product updates, engineering, our approach to AI from employees at DuckDuckGo. I’m Greg, I work on the search engine here at DuckDuckGo and with me today is Karl. Karl, would you like to introduce yourself?Karl: Yeah, sure. Yeah, so I’m Karl. I’m a design engineer here at DuckDuckGo and I work across the search engine on instant answers and our Search Assist AI module.Greg: Awesome. And that’s actually what we want to talk about today. DuckDuckGo Instant Answers, how we show instant answers for different search queries. Obviously, in a search engine, we show links to websites. But there’s a lot of other content that we show on the page as well, which I think broadly we sort of refer to as instant answers. You know things like if you search for the weather, you know, we might show a weather module at the top of the page. I’ll just share a couple examples of these. You know, so you search for weather and we try to show, you know, relevant information about the weather at the top of the page in addition to then all of the, you know, the links to different websites.Karl: Nice.Greg: We have things like unit conversions. If you search for a temperature and you want to convert it from Celsius to Fahrenheit, different calculations, all of these things are kind of within the umbrella of instant answers. We have had instant answers for a long time. I think really early in the history of the company and the product, recognized that when you’re pulling information from a lot of different sources, you have the ability to present that information in different ways to help the user find what they’re looking for, whether that is looking for a website that they want to navigate to, looking for an answer to a question, or looking for something a little bit more complicated. So yeah, we have lots of different instant answers, and thought we’d talk a little bit more about that today. So I guess as a starting point, I’m curious to talk about, and you mentioned being a design engineer, what are some of the challenges involved in designing and building instant answers?Karl: So I think one of the things that I’ve always found working on these is it’s very easy to just basically take everything we get. So we will work with like a provider. Let’s say if we’re doing where to watch something, let me share my screen actually as well and we can have a look at this. We are working with a provider. So in this case, we’re working with JustWatch and using their API data. When you come to design something like this, it’s very easy to look at that and say like, okay, cool, let’s go to JustWatch’s website and have a look at what’s available. We can see their website, how they present the information they have and they give us. And you could kind of jump to say like, well, we’ll just take what they’ve done. They’ve got that data, we’re using the same data, let’s just copy the way that they’ve presented that information. That would be like the easy solution. But then you’ve got all of these sort of legacy things to deal with, like we talked about a bit earlier, we’ve had instant answers for a long time. We’ve got ways that we present our information on the search engine and we need these modules, whether they’re pre-existing or they’re new, like we need them to all feel like they belong together in some way. So it needs to sit in the search engine nicely, you know, and feel kind of like it belongs. So we can’t just do like a one-to-one copy, which, you know, as designers anyway, we don’t want to just copy what someone else has done. But when the data is so vast and so complex, it can be kind of hard if you’re not familiar with the domain so much as an individual to just figure out like, what do I even care about here? So like, here’s an example with a movie, we were trying to figure out where you can watch this, but like, where are you in the world? You know, because that’s like the first thing that’s probably one of the biggest challenges is we can’t show you the same thing if you’re in the US or the UK or Australia. You’re going to see something different.Greg: I think you touched on a lot of important pieces to this that make this challenging. One is just understanding the user’s context. Obviously, we have limited information about our users based on our privacy policy and our privacy standards. We can approximate things like location accurately enough to be able to do things like the weather one that I showed, or this where to watch example. But that does limit us in what we are trying to do where we’re not trying to overly personalize the things that we’re showing down to a very specific user profile. And so, we have to kind of generalize and try to figure out just from the search query and that limited bit of information, what is it that we’re going to show to try to answer the user’s query. The other thing you touched on that I think is important to highlight is just the fact that this search query here is very specific and I think has a pretty clear intent where to watch a specific movie. Sometimes the intent is a little bit harder to unpack. And so you might have more different things that you could show. If you just searched for the movie title, for example, maybe someone’s trying to figure out where to watch it. Maybe they’re interested in the release date or different news about it. If it’s an upcoming movie, maybe they want to know what the cast is like. So trying to figure that out and then present the information in a way that gives users the ability to see what they’re looking for and also then refine their search down to the more specific thing. I think that adds some challenge as well.Karl: No, I agree. I think probably one of the biggest things that I found difficult around this is exactly that is the like figuring out the query, figuring out the query space, and then defining the data set, the golden data set, as we call it, that helps us to understand when do we trigger? How do we decide what we trigger? And entertainment is especially challenging because we have three different modules that are part of the class of entertainment. We have the titles module, which is the overarching piece that tells you, you know, this movie is from this date. This is a brief synopsis and you can get to the two other modules. So this where to watch module. And then also you can get to the cast module from that parent module. But like you say, you have to trigger these differently and figure it out. And so we can do these really clear ones where someone puts the keyword in that we’re looking for, but sometimes we’ve got to try and use all these other signals from what we get from the organics or like other pieces of information that we’re retrieving to say like, actually we think this is best and you will know better than I, but like it’s incredibly hard and sometimes, you know, it doesn’t matter how hard you try, the intent might feel really obvious to the person typing it, but as far as we can derive from that information, we can’t always figure out the right thing to show you. And so we do our best, but you know, there’ll be misses sometimes.Greg: Yeah, I want to ask you about something that’s on your screen right now because you can see, you searched the query Project Hail Mary cast and we’ve got the cast module at the top left there. We also have that Search Assist box to the right side that attempts to kind of answer the question. I guess, we’ve talked a little bit on previous episodes about Search Assist and what it’s doing. You know, how would you describe how we think about showing those two things next to each other, the different kind of roles they play, and maybe how that’s evolving over time?Karl: I think, yeah, that’s been a big challenge for us because AI summarization is a really useful tool for some people. It’s not for everybody. And obviously, we’re super careful about giving people optionality and letting you say, I don’t want to see this. But for people who want it, people who are interested in seeing that, we don’t want to just always put that front and center as the key answer to every query. Because in a case like this, for example, Search Assist is giving you the same information, but it’s giving you it as text. And maybe we’re giving you a little bit extra too. But if I’m specifically trying to ask about the cast, there is a high chance it’s because I’m thinking of an individual who I’ve seen in the film, or I’ve heard is gonna be in the film, or what have you. And so I’m trying to see their face to be like, yes, okay, so it is Ryan Gosling who’s in Project Hail Mary. Okay, cool. And so in this case, like that is probably what we feel the most critical piece of information. So we want to give you that module upfront. But at the same time, if you’re someone who’s interested in getting a kind of AI summary as part of your query, we don’t want to then say, just, we won’t show you that tool then. And so we have this like convenient little right rail slot where we can say, actually, we’ll just move this over here. So you get kind of the what we think is the best of both worlds where we can deprioritize Search Assist in this case where the intent is really clear. And then in cases where we’re not so sure, we will do our best to try and figure out like maybe Search Assist is the best answer of this or maybe it’s a different module.Greg: Yeah, I’ll also note, we have different sort of settings that you can use for Search Assist everywhere from, I never want to see Search Assist at all up through, show it to me as often as you can. You obviously have it set to show often here. I do too. But yeah, I mean, for this kind of query, certainly the use case you spoke to of like, I’m sitting with my family watching a movie, go, wait, who’s that? This is the type of thing where, okay, you can get that information really quickly. But I guess to your point, we recognize with a search engine, you’re walking a line between giving just the immediate information that a user is asking for and then giving them the ability to dive deeper, which a lot of people are using a search engine for. It’s sort of a, people come and type all manner of topics and things into that search box. And we kind of have to be ready for any of it and different user expectations. I would say we do, I’m curious to the extent to which you agree with this, like we do generally try to keep things relatively uncluttered, like we’re not trying to sort of throw everything at the page all at once. You know, I think from my perspective here, this is sort of one of the more kind of rich search results pages you would see on DuckDuckGo. You’ve got the news module down below. You’ve got some kind of imagery within the organic links. And that’s about it. You know, I guess from a design standpoint, how do you kind of see the role of just like the information layout and kind of how much we’re showing on the page at a time.Karl: Yeah, that’s a good question. I’m a huge clean freak when it comes to design as well. It’s kind of hard trying to walk the line because when we talk to people who use the search engine, we’ll try to understand, how do you feel about the page they’re presenting? Especially if we try and compare to anyone else, what do you think about it? And we always get kind of mixed results, I guess, because some people love a nice clean layout. Some people want to see loads of stuff because they’re just so used to being bombarded with richness of like images here, there and everywhere. And one of the biggest challenges is that we obviously want to meet people where they are. We want to help give them what they’re looking for, but we don’t want to go to the kind of degree that we just say like, we’ll absolve all responsibility and just give you everything that’s available. Like here’s every image and every kind of video and stuff’s auto playing and like just taking up all of your space. Like for me, like you said, this current SERP that we’re looking at is this layout of the page. Like this is very rich for us. And it’s like at that kind of point of balance where I think we are okay with it. And we’re like, that’s good. But you know, if we imagined that Search Assist also had an image at this point as well and other things in there, it starts to make it really difficult for somebody to figure out like, where does my eye need to go, what do I need to look at in this particular point? So we’re trying to kind of keep it really clean and say, you know, cast is this most important piece right now. That’s you’ve got big images, big module, and then we kind of de-prioritize as you go further down. But it is probably one of the hardest challenges and it’s why whenever we’re developing a module like this, it’s not the kind of thing that we can develop in isolation. You know, you can’t sit in a Figma file just with this design of a flat version of a cast module and be like, you know, that looks great. I love it. Because the minute you put it in the SERP, you realize, you know, okay, I’ve got to deal with all of these other visual things and maybe we need to turn some stuff off or change something. And that’s I think, you know, being a design engineer here, it made it possible when we developing these entertainment modules for me to do that. So I was given the kind of the rope to get into the code base and start to put these things in a SERP and play with them. And I guess you just, you get a different feel for what works and what doesn’t work when you actually see it in real place in the code base.Greg: Yeah, definitely. I’ll also add, similar to the Search Assist setting, we have a setting that lets you basically turn the instant answers off if that’s what you want to, recognizing that some people like more and some people want less. Although I think in a lot of cases, we see users keeping them on and finding a lot of value in them. I’m curious, as we maybe get towards wrapping up our conversation, putting your user hat on for a minute. What are the instant answers that you use the most?Karl: Well, I will say that I use my entertainment ones all the time, of course, but that’s also just as a kind of, I guess, a curious person who works on them. I have to keep an eye on them. Search Assist is obviously one of my, again, most frequently used, but I think conversions is a surprising one that I use a lot, and I think that’s especially the case because being from the UK but working with a lot of my American colleagues, when people are just chatting about Fahrenheit, I have no idea. So I have to end up getting in there and converting that. But also great for time zones and things like that when you’re trying to figure out when you can do a meeting, you know, like this meeting, for example, I can jump into that. And then weather, again, like one of the simple things that just, it saves a ton of time when you’re already in a search engine doing other things, just pulling up the weather. IA is a super handy one. What about you? I feel like you’re gonna be a sports fan.Greg: Yeah, well, we haven’t talked much about sports, but I do use the sports modules a lot. The sports ones are interesting because, you know, with something like conversions, you know, that’s pretty consistent as far as the queries we see day in, day out, people like looking for conversions. Sports is going to depend on, you know, the events that are happening, right? Our American football module is probably not triggering very much right now. But when the season starts in the fall, it’ll start showing again. And certainly, the day of the Super Bowl, it’ll be shown quite a lot. I’m a big Philadelphia sports fan. I have some queries bookmarked that I just look at pretty frequently. I find, personally, the instant answers for sports are actually kind of my favorite way to get some of this information. You know, if I’m looking for like the NHL standings and, you know, or kind of, you know, when do the Flyers play their next game or when did they play their last game? Like this search query is really quick for me to do. It loads quickly. I kind of know what all the information is here and where it’s going to be. So I find this pretty handy. And we’ve actually just this week, I think we’ve shipped some new sports. We’ve shipped Formula One, we’ve shipped NASCAR, and we’re planning to add a bunch more sports as well for people who want to get their sports information from DuckDuckGo. So yeah, I guess anything else we missed that we want to talk about before we wrap up?Karl: I think we’ve covered pretty much everything, yeah, I can think of.Greg: Yeah, I’m sure we could do future episodes deep dive into more. We didn’t talk about local at all, which is its own answer that has a lot of complexity to it. But yeah, I guess we can wrap up there. Thanks, Karl. This was fun.Karl: Yeah, awesome, thanks.Greg: All right, bye. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit insideduckduckgo.substack.com

In this episode, Peter (Product), Stephen (Design) and Balint (Windows) discuss updates to our browsers, from the most popular custom themes, to why over 40% of our iOS users have made DuckDuckGo their system-wide password manager. Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.Peter: Hi, and welcome to DuckTales, where we go behind the scenes at DuckDuckGo and discuss the stories, the technology, and the people that help build privacy tools for everyone. In each episode, you’ll hear from DuckDuckGo employees about our vision, product updates, engineering, and more recently, approaches to AI as well. My name is Peter. I am on the product team at DuckDuckGo, often working on our browsers, which we’ll talk about today. And joined here today, I have my colleague Stephen.Stephen: Steven, work on the product design team.Peter: And Balint.Balint: Hey, I’m on the Windows Developer Team.Peter: Awesome, and today we’re going to talk a little bit about our browsers, as I mentioned. So many people don’t know we have browsers. They’ve come to know us through DuckDuckGo Private Search, which of course you can use in any browser. But we’ve for many years offered browsers on iOS and Android devices. And more recently, in the last few years, we’ve expanded that to Windows and Mac desktop computers as well. Our browsers are amongst the most popular in the market. For example, in the United States on iPhones, we are the most popular browser after Safari, which comes built in, and the Chrome browser, which of course many people have come to know over the years. So our browsers are used by lots of different people. When we started building our browsers and as we expanded our browsers to desktop computers, we started to prioritize the features and functionality in our browsers based on sort of two things. One, having privacy at their core. Of course, that’s our brand promise, and we want to deliver on that. But also, we wanted to make sure our browsers were easy to use replacements for the privacy invasive browsers that people have been using for many years. And so we prioritize a lot of features based on what would make it easier to use. We base it on people’s feedback, which we’ll talk about a little bit more. And often internally, we talk about focusing the features we build on the three Ds. We call that dependability, discoverability, and delight. The features should just work. It should be easy to find. And they should be delightful if we’re doing them right. So today, we’re going to talk about a couple different browser capabilities that fall into a few of these buckets that we thought would be interesting to share a little bit about. First, we’ll talk about custom theming. Stephen, do you want to tell us a little bit about that?Stephen: Yeah, custom theming to me is pretty simple. It’s where you can go into your browser and just choose which color you want to use. It can be that green or blue or purple. I think we have a range of colors to pick from.Peter: And why did we build custom theming?Stephen: Originally, we didn’t plan to. We were going to try to keep it simple and just have a light and a dark mode. But we got into it we discovered that people really like to customize their UI. It’s just like a fun way to make browsing more fun and personalized. And they really like that. And it also can help you pick your browser out from other browsers. So it’s really also a functional improvement.Peter: Do you want to give us a little bit of a demo on how that works?Stephen: So we built two ways to get into this. If you’re on the new tab screen, you can open the customized sidebar and we have a theme picker up here with a few colors. You can pick gray or blue or green, purple. Lots of people love purple. And you can also change the theme from light to dark or just to use whatever the system decides. So we support all of these. And if you want, you can also change the new tab background screen to be separate from the rest. And we’ve spent quite a bit of time making sure that this works on all surfaces. So you can see it in the settings and the bookmarks in the history and elsewhere. Also, we have a neat little feature here where you can change the app icon and the dock to match the theme. So that’s a little more continuity and a little more fun added to the feature.Peter: Stephen, were there any challenges in building this as we started to introduce it?Stephen: Yeah, so like I said, we didn’t really have themes on the roadmap when we started. So we kind of had to go back and rethink the way we were styling things and build a whole theme system so that we could apply a bunch of different themes and make sure that it was, we could have more themes in the future. So now it’s all centralized and easy to work with. So we could probably add more or maybe even enable customization at some point, user customization.Peter: And would you say, what’s the response been from our users so far?Stephen: It’s been pretty positive so far. We got a lot of feedback saying, thank you for adding this. And it’s been pretty positive and monitoring people using it. It’s going up. So that’s good.Peter: And you’re saying the certain colors were the most popular, I think you mentioned earlier.Stephen: Yeah, so I think the most popular color is, let’s see, it’s sleet blue, I think. And then the second most popular color was violet. It’s kind of neck and neck there. Personally, desert is my favorite, but it was not the most popular.Peter: Got it. Cool. Where are we going next with it?Stephen: We’re looking into adding a dark mode for web pages that don’t have it, which is going to be pretty interesting. And like I said, we may, depending on if we get more feedback on how many themes we have, how many colors we ship with by default or expand that out to letting people choose their own colors.Peter: Yeah, the dark mode for sites to use dark mode, that’s starting to roll out. We have that on mobile devices, on Android starting to roll out on iOS. And it’s something, as you said, we’re looking at on our desktop browser. So I think our users can look forward to that in the near future, which is awesome. Great. Let’s talk about password management next. Maybe Balint start with what is a password manager? Maybe people don’t realize that in their browsers they have typically password managers. Maybe just grab that at a high level first.Balint: So the password manager is the thing that comes up when you go to a login screen and the browser offers you to fill your username and password automatically. And it’s a critical piece of feature because one of the most dangerous attack vectors used to be credential stuffing where an attacker got hold of your passwords from one site. And if you have been reusing the same password on other sites, they could just brute force. They could just try the same password with your username on other sites and usually get a hit. And the password manager is a perfect defense against this sort of attack because it takes the burden of having to memorize passwords off your back.Peter: That makes sense. Yeah, I guess, you know, over the last 10 years, a lot of you hear a lot about third party password managers like 1Password or Bitwarden or LastPass and more people are using those. But I think it’s probably safe to say that most people use their browser as their password manager. Maybe they don’t even recognize that it is a password manager, but it is, as you’re describing overall. Why did we choose to sort of build it into the browser and start to build out a lot of those capabilities? Is it because that is the easiest way for people to protect themselves, as you’re describing?Balint: I think it matches perfectly well with the 3Ds that we have. So it’s a core part of dependability. You want your online experience to just work, not have to memorize passwords and not have to think about where those passwords are stored. Of course, if you explicitly want to, you can use a password manager, but it is very nice to have a secure default built right into the browser. And we take care of the whole lifecycle. So when you register to a new site, our browsers will suggest secure passwords, which are randomly generated and long enough, take care of storing them, and if you turn on sync, can get your passwords synced across your different devices so you don’t have to re-enter them on every single device you have.Peter: And we’ve had the core capabilities in our browser product for some time. What’s been the user response and maybe describe where we’re going with it next.Balint: So we see on our anonymized dashboards that uptake for these password managers is very high and we keep adding new features. For example, recently over the past months, we have shipped a feature for Android and iOS devices where you can use DuckDuckGo’s password manager as the main system password manager and you can fill passwords in other applications from the DuckDuckGo database. And uptake for that feature has been phenomenal. We have almost half. So a bit more than 40% of our iOS users enabling us as their primary system-wide password manager. We have more cool features in the works. My personal favorite is something that is called TOTP. It is that six digit number you see on certain websites, which is used as a second factor authentication. This is something that many password managers already offer, but browsers don’t really tend to. We would be one of the first on the market to ship this. And it would present a second security layer that is much harder to phish, so much harder to steal than a conventional password.Peter: And do you think we’ve hit challenges in building this along the way? And what are some of those challenges?Balint: Certainly. So password management is all about security and figuring out a way that lets us store the information on the devices securely in a way that we can access them, but an attacker cannot access them. And we have been very proactive in using operating system level security features. For example, on Windows, we are storing them in a way that is secured by Windows Hello. So that is Windows’s own built-in encryption mechanism. So breaking our password manager is similar in difficulty to cracking Windows’s own encryption, which makes it a pretty hard target.Peter: That makes sense. So when a user goes to access a stored password, that’s why the browser will cause Windows to trigger the Windows Hello authentication overall. Makes sense. And then you talked a lot about storing all these passwords. And, you know, as I mentioned earlier, a lot of people don’t know that we even offer browsers to start with. And so if they come to us and install our browser for the first time after many years of using another browser, presumably they’ll have a lot of passwords in that other browser that they need to get into our browser somehow. Maybe we can talk about that and how we’ve been iterating on the import capability, so a little bit.Balint: Exactly. So we offer an import capability which you can do as part of onboarding. So when you first come to our browser, one of the onboarding steps is the offer to import your stuff, but you can also do it later on from main, from more tools and import on Windows. Similar logic is available on our other platforms as well. And one of the recent additions to this has been the so-called multi-import where you can very simply select not just one target or one source, but several of them, we can import directly from the most popular browsers on the market. And it’s been trimmed as well. So if everything goes well, it is just one or two clicks and you are done. We support importing passwords and also bookmarks. And this also sees pretty good uptake. So once we have updated the visuals for this multi-import, we have been seeing 13% more people choosing to import and over 70% more stuff getting imported, so 70% more passwords and bookmarks. And that is pretty good news because it reduces the friction. So when you come to DuckDuckGo, you don’t have to restart your digital life from scratch. Instead, you can continue from where you left off, except in a more private manner, of course.Peter: That’s awesome. Yeah, my experience is, you know, over the years, people tend to often use more than one desktop browser, sometimes one for work, sometimes one for personal or for separation of different context. So the fact that we make it easy in one step to import from multiple at a time seems to be really beneficial and delightful to users overall. That’s great.Balint: Yeah, and we are also innovating on our mobile platforms as well. So for example, we have been first to market with a feature that lets you import from mobile browsers directly. So not via a desktop browser that gets synced to your mobile version as well, but directly on the mobile browser itself. And we are planning to take care of the newly released iOS API, which will further streamline imports on the iOS platform.Peter: That’s great. Yeah, we have that both on iOS where you can import from Safari and then we also have directly imported from the Google password manager on Android, which is great for people. Awesome. Well, thank you, Balint. Thank you, Stephen. I think as a summary, what I’ll say is if you are using our browsers or if you’re not and you choose to start to use them, we’d love to hear your feedback. We do very much listen to user feedback. There’s ways right in the product to send us feedback. You can fill out the form. We absolutely look at those, read those. If you go on our subreddit and discuss our browsers and capabilities there. We absolutely look at those threads and we take all of that as input to help make prioritization decisions about the features we’re building. So thank you very much and see you next time on DuckTales.Balint: See you. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit insideduckduckgo.substack.com

In this episode, Gabriel (Founder) and Caine (CTO, first employee) discuss the history of our search engine, why now is the right time to build a full web search index, and how our scale makes us uniquely positioned to ship, learn and iterate quickly. Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.Gabriel: Hello, welcome back to Duck Tales. I haven’t been here in a while. And I am Gabriel Weinberg, the founder of DuckDuckGo. And I have with me someone who I don’t think has been on Duck Tales at all yet, but you should know, Caine Tighe, who I know very well, who’s the first employee of DuckDuckGo and now our CTO. Caine.Caine: Hi Gabe.Gabriel: We’ve been working together for a very long time. And we’re here today to talk about something we’ve both been working on. Caine more than me, but I’m working on it some, which is our web search index. So as some background, first some background. DuckDuckGo started as a search engine, as many people know, and it was actually started by me. I was by myself for a few years. And the first thing I did was start crawling the web and building a web index.Caine: Yeah, for sure.Gabriel: But you know, I soon realized that that is very expensive, especially as one person. And there were other places to get a web index at the time. And what was more interesting was maybe adding value on top of the web index. So building other indexes, this was a time, this is the mid 2000s, you know, there weren’t, there obviously wasn’t AI, but there wasn’t even really many instant answers on search engines.Caine: I mean, that’s what we were working on together at the very, very beginning. Like we were working on, you know, you had the knowledge graph. It wasn’t called a knowledge graph at a time, but you were doing all the structured content from Wikipedia and otherwise. We worked on some other smaller indices. So yeah. And then actually fun fact, in hiring our backend project is still based on some of the original spam and content farm crawling, like one of the projects is based on some of the spam and content farm crawlers that you originally wrote. So that lives on 15 years.Gabriel: Yeah. So we were doing lots of indexing and lots of crawling. Yeah, exactly. Just not, you know, we started, but then we stopped doing a full web index, but just as examples, right? We started like the code that you were talking about indexing Wikipedia, which became our knowledge graph, you know, which is, powers a lot of answers, which also we used when we started working on AI answers. We’ve been doing local indexing for, you know, over a decade, local businesses and things like that. You know, then all sorts of kind of niche indexes that involve some crawling like lyrics and things like that. So indexing technology is not new to us, despite what some people say about it. Sometimes we do lots of search indexing, but we hadn’t been doing a full web index until relatively recently, last few years-ish. But now we are. And so the question is, the questions and why you’re here, and we’ll talk about it for a few minutes, is kind of why, what’s going on, how, all the main questions, which we’re obviously not all gonna answer today, but we can start with, I think, kind of the why, but why are we well positioned to work on this? So to speak, and you’re kind of at the center of it, so I think you’re a good person to ask.Caine: Yeah. I mean, I think, the why now is a mixture of like our needs. Like we want to support our own AI use cases. That’s we have two primary agent, agent driven products out. Search Assist, which is on the SERP, search engine results page, duckduckgo.com. And then we have Duck AI, which is our chatbot. And both of those products are hungry for this kind of data. So it, yeah, it just makes sense for that.Gabriel: Yeah, in particular, right. You could maybe talk percentages, but like there’s some percentage of search results now, what is it like 25%, I think that have Search Assist answers. And then, you know, the percentage better made for Duck AI, but some significant percent call the web, 15% maybe.Caine: Yeah, I always do. I do, I have my numbers based on, absolutes make more sense. Yeah, yeah, just, yeah.Gabriel: You know what? Bad question. Ignore numbers. Doesn’t matter. Good percentage of queries and Duck AI prompts require web search. And so we need a web index for it, essentially, right?Caine: Yeah. I mean, I think on the chatbot side, it’s really good, like to ground. If you’re deciding whether or not to ground and you’re on the line, you should probably use RAG, retrieval augmented grounding, and go out to a third party data set. For us, raising the standard of trust online. We want to do that because the more that you ground, it’s known empirically, the answers are better. So we err on the side of grounding where I think maybe not everybody does. So it’s really good that we need to build our own index in order to be able to accommodate that. So that’s kind of some of the why now. And again, it’s on Search Assist and it’s on Duck AI. One of my favorite parts about this whole thing is like, we’re very used to working for customers, like our end user. For the search index, duckduckgo.com itself is the customer. A very nuanced, unique thing for us to be able to serve ourselves, which creates this really tight feedback loop internally. So it’s been cool to like use our own and we are live for, you know, some amount of the traffic today. That’s just growing day over day for these use cases.Gabriel: That’s a good point because like, I think in terms of like our position, well, positioned to do it is, you know, being live, you know, maybe we talk about that a little bit, but like that creates a feedback loop that we have that a lot of people don’t have because we have, you know, many, many millions of people using our search engine and now Duck AI. And so we’re getting constant feedback about the relevancy of the search results that we’re serving, not to mention the fact that we have almost 20 years of evaluating relevancy ourself on our own search engine.Caine: Exactly. Yeah, exactly. So like, humans are unsurprisingly and appropriately more, more critical of results than agents are. So it kind of creates this higher fidelity feedback loop because, you know, through our, through anonymization and whatever else, like we can privately understand what is most relevant on the internet for customers and users. And that really helps us to, positions us to be pretty competitive in the space quickly. So like, I think that’s kind of interesting and it’s exciting and like the true DuckDuckGo way as you and I know well, like we like to ship stuff. And so it’s been really cool to, yeah, like it’s just been really cool to be using it already and in production, our own index. And it creates that flywheel and we could, you know, use buzzwords like reinforcement learning and this, that, and the other thing. But at the end of the day, it’s just really the relationship of consuming your own internal API product. That’s the flywheel and allows you to like establish relative priority really quickly and be like, I ran this experiment. Like we really think this query set is going to be well suited to our own index. And then we tried and we’re like, we’re not working that well on that. Let’s move to this other one. And then it just changes the game for how quickly you can iterate, which has been really exciting for, and I know the team’s really excited about it too, because engineers like to ship things. So that’s been cool.Gabriel: Perfect, I think that’s a good intro. But let’s do, to your point about buzzwords, let’s do a few more buzzwords in terms of like, just give us the broad tech flow, like, you know, without getting too deep into anything, but just to give people a sense of kind of how it works and then maybe.Caine: Yeah, so kind of the way that I think about this is like a little pipe or a train or whatever. You have your frontier that kind of is the web that you’re looking to crawl, like, because you have to pick what your frontier is. Then you crawl that. Crawling, all of these components are extremely complicated by themselves. To crawl, it means like, you need to crawl politely. Some sites want you to crawl, some sites don’t want you to crawl. And so to be a good trustworthy netizen, you have to respect those things. And that’s an important part of crawling. It’s also important to have the bandwidth and the throughput to crawl at the scale that you need to crawl. And so fortunately for us, we’ve had a lot of experience with that, so we have that. The rendering side, you have to, when you fetch content, you have to render, including JavaScript and everything else like that. The only way to get the content is to literally run the whole webpage. Otherwise, like you get no content. So that’s quite an expensive process. So we kind of do a naive approach and then a more complicated rendered approach. Then you have content extraction, which is like the next step, where you think about your title, your description, your headings, metadata, main body stuff, where you extract the content, what the page actually means. And then we’re very fortunate in today’s day and age to have semantics. So semantic search is a big part of the pipeline. And what that means is what people are calling embeddings. And you calculate embeddings on extracted content. And then we use a database, which I quite like, called Vespa. And it’s all ingested into Vespa. In my opinion, kind of your indexing, your ingestion, your features, how you calculate those things, and how they describe, that’s a big description of your product. Because it limits what you can do in the ranking phases, which we could get into. I don’t know if you want to get into the ranking funnel as well or just the data pipeline.Gabriel: I think that’s a pretty good overview. I mean, I think, we’ll do more of these in the future, but like, if people have questions, they can write me or write us, but we need to get an email going where people, we have the proper feedback for this podcast.Caine: A mini series as it were. Yeah, because I’d like to bring in, you know, I can give you the overview, but there’s a lot of experts on the team that, yeah.Gabriel: Yeah, so I think what we plan in general to come back and we can talk deeper about specific things in the future, but I think that was a great overview. I think we hit it. Anything that we missed that you want to cover or we’re good?Caine: No, I mean, I think the space is quite interesting and it’s been really fun. Like when we started out, you know, I started out with you 15 years ago and like, we were doing something different back then. The concept of a customer being an AI agent was not really a thing. And I think AI agents as search customers is quite interesting because like, they generate the queries. AI agents are generating the queries. They’re there. The queries are formulated quite well. They don’t feature misspellings. AI agents can consume large amounts of information quickly. Like lots and lots of information. They can speak in embeddings. So I just think it’s a really exciting time. And I think we made the right call to kind of forgo it until more recently. Because now we’re quite well equipped to support ourselves. And we have the products out that really express the needs and the requirements of an API like this. So I know I’m very excited for that, and the team’s very excited for that. So yeah, we should do more of these and get deeper into what a polite crawler really means, as opposed to just glossing over it, because it’s important.Gabriel: Cool, well, thank you, Caine, for joining, of course. We’ll do more again later. If anyone’s got questions, email them in to us. Otherwise, see you till next time. Bye, everybody.Caine: Later. Thanks. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit insideduckduckgo.substack.com

In this episode, Beah (Chief Product Officer) and John (People Ops) discuss aynchronous working, no meeting days, and the role of face to face meetups. Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.Beah: Hello, and welcome to DuckTales, where we go behind the scenes at DuckDuckGo and discuss the stories, technology, and people that help build privacy tools for everyone. This is my dog, Friday. He’s appeared in other episodes. So in each episode, you’ll hear from employees about our vision, product updates, engineering, or our approach to AI.John: Hello.Beah: In this episode, you will be hearing about our approach to remote work. I’ll quickly introduce myself. I’m Beah. I am on the product team at DuckDuckGo. Then, John, I’ll let you introduce yourself.John: Thanks, Beah. Yeah, my name’s John. I work on the People Ops team at DuckDuckGo. You can probably hear from my accent. I’m one of the couple of members of the team that’s based in the UK. So my job is predominantly around meetups and a little bit about culture, but predominantly about meetups and how we meet up in person and a little bit virtually as well. So yeah, that’s my role here.Beah: Great. Yeah, so that is a good segue to jumping right into talking about how we work remotely and how we connect personally given the remote circumstance. So maybe just before we get too deep into the details and what we actually do, can you describe, John, just like what we even mean when we say that we’re a remote company?John: Yeah. So in very simplistic terms, I would see being a remote company as us not all going to one central place to work. You know, we don’t have a big office building that we all come into either on a semi-regular or, you know, an everyday basis. So that is people probably predominantly working from home, but not necessarily. We have ways and means at DuckDuckGo for people to work, you know, in a co-working space with other people if they need that or if they have circumstances at home where they need to. Essentially, there’s a choice and a trust in where people work. We feel, you know, there’s pros and cons to that, but we feel there’s a, you know, overall a net positive, I think, to that way of working. So that’s the way I would think about it, that we work in that way. And then to facilitate that happening, you know, all companies will have tools to, you know, online digital tools to allow people to collaborate and move their work forward. But we maybe think a bit more or maybe have a few more tools that allow us to collaborate digitally and make sure that we do what we need to do online essentially and digitally.Beah: Do you know, you might not, but do you know how many people do work from co-working spaces or somewhere social that is not their home?John: Yeah, not too many in the company. So we offer a financial element for everybody if they want to work in a co-working space. I think we definitely have a handful of people, certainly in our organization, who pretty permanently work in co-working spaces. And that may range from family circumstances or living circumstances, whether it’s just difficult to work from home, through to people that just need that kind of social interaction each day. And that’s what makes them more productive. I think there’s a handful that work fairly permanently from coworking spaces. And then there’s definitely a good chunk of the organization that will treat themselves for a day in a coworking space, maybe once a month or meet up with somebody to work with. But yeah, I’d say that most of our team day to day will work from home, really, have a set up at home. Yeah.Beah: Yeah. And I guess, I mean, we have, maybe I’m getting ahead of the conversation, but we also have plenty of locations where there’s like clusters of DuckDuckGo people, like a dozen people or five people or just three people, and they will sometimes get together, either like work together or just like grab dinner, grab lunch, right?John: Yeah. And that’s happening more and more as we, you know, as we get bigger. You know, I had a quick look at our stats before we started this conversation, you know, 10 years ago, we were around 30 people. So, you know, meeting up was much harder. Now, you know, 10 times that amount. You know, for example, I know there’s a meetup happening in Spain. I can’t remember if it’s Barcelona or Madrid, but there’s a meetup happening soon just because we’ve had a lot of new starters start in that region. So somebody thought, well, that would be nice. We can get to our different types of meetups and how we arrange that. But that’s really cool. We’ve had a few people start in Spain and someone’s thought it’d be really nice to meet up in person. So yeah, whether that’s sometimes dinner or just a co-working day, we’re becoming more and more common.Beah: Okay. I actually am in the midst of planning a family vacation to London and we have a ton of folks there and I am gonna try to plan a meal and see as many people as are willing to come meet me for a meal.John: Yeah. You see, that’s nice. That says something that you don’t want to have for your holiday and completely not see anybody from work. That’s kind of nice. There will be some people who don’t want to do that. But yeah, that’s really cool. That’s really nice. Yeah, we’ve got a lot of folks in London.Beah: Yeah. Yeah, I actually like that. It’ll be cool for my family members to meet folks too, because they, especially as we’ve gotten bigger, they know fewer and fewer of the people that I spend my day with, and so I’m excited.John: Yeah, yeah that’s nice, that’s really cool.Beah: Okay, all right, I kind of got us a little off track there, but I think that’s okay. So tell me a little bit about like, you know, given that we’re just everywhere in the world, how do we do meetings generally? What kinds of meetings? How do we talk live to each other?John: Yep, no, don’t worry. Yeah, I mean, taking a step before how we do meetings, I would say we make an effort in some ways not to do meetings if we can, let that go. So another way of thinking of working remotely or as we, it’s a bit of a pretentious term, asynchronous working or async working. I know when I said that to my family, they were like, what? I was like, essentially we work online, but we write a lot of stuff down. That’s the way I would think of async working. We maybe write more stuff down than a lot of other companies would. So I would say we don’t try and avoid meetings for the sake of it. But I would say that if we can, we try to do things async if we can. And even if we do have meetings at DuckDuckGo, we try and do as much prep before that to save as much time in those meetings as well. And I know I found it slightly disconcerting, but also amazing when I joined DuckDuckGo that sometimes we had a meeting in the diary for half an hour and in previous organizations, half an hour wouldn’t have been enough to cover it. But not only that at DuckDuckGo, we finish the meeting sometimes in like nine minutes because there’s been a lot of chat before the meeting and you sometimes feel like, well, this feels a bit too easy, but it’s because that work has been done already and the meeting is just a really important thing for us to align and if there is anything else. You know, we, other people have maybe talked about this on this, we, Wednesdays and Thursdays are non-meeting days for us in terms of standing meetings. You know, we try and keep those, well, we do keep those for deep work. We don’t make exceptions to that rule. So we do use Zoom, you know, for, and we have, and I really like this, we have a number of processes, I guess, whether that’s kicking off a project or post, what we call post-morteming a project where we, I would go so far as to say we mandate, don’t we, getting together in person, we think getting together on Zoom is important to do those meetings. And we don’t make exceptions to that. And that’s what I mean by having sometimes a very quick meeting, we decide what processes require a meeting. So yeah, most of them done through Zoom. I think it’s very rare at DuckDuckGo to have really more than a half hour meeting, isn’t it? For most project kickoffs and post-mortems team meetings, maybe a little bit longer, but even then we pack quite a lot in. So yeah, most of our meetings are done online and we try and keep them as minimal as possible and as useful as possible. Yeah.Beah: Yeah. Yeah, I mean, the way I think about it is like, we, I feel like we try to, you know, reason from first principles about when a meeting is the right venue for something rather than defaulting to meeting. And so there are times when that is absolutely critical. Then there, and there’s times when it’s actually like not the best way to make a decision or come to some conclusion or a hybrid is the best model, like you said, of thinking it through, writing it down ahead of the meeting, meeting to pull out the nuances, do the things that happen really well live. And then also we try to like, you know, we kind of have a principle of like a decision isn’t really made in a meeting. Like a decision can kind of be discovered and cultivated and then we kind of like write it down because there are times when you’re like, you’re caught up in the moment of the meeting and maybe the social dynamics or I don’t know, you’re just, you’re thinking on the spot and then afterwards, you know, in a moment of reflection or writing like, you think, well, maybe that’s actually not the logical answer.John: Absolutely, yeah. And I think another thing I found about meetings at DuckDuckGo, you know, I’ve been here about three years and this is no slight to any other organization I’ve been at, but it also feels slightly disconcerting not giving really any status updates at meetings. Like we tend to dive straight in because we assume that everyone attending that meeting is aware of the things we’re going to be discussing at that meeting and is relatively prepped for it. We know everybody’s busy, but relatively prepped for it. And as you say, it’s almost like converging on a decision and that can feel slightly strange compared to other organisations where you are sometimes in meetings spending your time getting people up to speed. I don’t think we do that a lot at DuckDuckGo. We tend to all enter a meeting pretty much knowing why we’re there, what our role is in that meeting, why we’re there to discuss what we’re discussing, you know. So that feels great as well. When we do have a meeting, it feels purposeful in that way. Yeah.Beah: Yeah, I mean, okay, so maybe in contrast to the somewhat throwing shade on like spending too much time in a meeting, there is value in all being together, you know, via Zoom and in person. Let’s talk a little bit about the in person part or a little bit more because we kind of touched on it. But what do we do in terms of getting people together?John: Yeah. So we, this again, when you are a company that is evolving and growing at the, you know, there’s companies that, you know, will grow people wise a lot quicker than us, I get that. But when you, when you do sort of grow tenfold, that can be, feel quite dynamic in terms of how we do it. So that is still constantly changing and we’re working out what works best for us. As a principle, everybody will meet up in person at DuckDuckGo generally twice a year as a minimum-ish. So our functional teams all have an opportunity to meet up once a year somewhere in the world and they will get together. And then we have an all company get together. Yeah, good point. So we work across objectives. SoBeah: What do you mean when you say functional teams?John: I know other people have discussed this in this podcast series before, but we work across objectives rather than always in those functional teams, but we do have a home. So we have, for example, a design team, a certain engineering team. You’re in the product team, I’m in the people ops team. So we will have functional meetups once a year with those teams, me and my, there’s nine of us in the people ops team. So we will get together once a year. And then we have an all company meetup, which is again, once a year, we hold them at a cadence where they’re roughly six months apart. So everybody can sort of see somebody, see people in real life every six months at least. So they’re the two main ones that we do. We do co-working meetups as well, which is a lot of fun. So anybody can suggest a co-working meetup. We don’t cover absolutely everything for that, but we give people the opportunity to book office space and hotels, et cetera, to meet up as a co-working meetup.Beah: Okay.John: And then aside from that, and this is another thing I really like about DuckDuckGo, we have a high level of trust to then allow people to meet up really in any way they feel is beneficial. And we have checks and balances on that, but most of our objective work is done online and is done not in person, but every now and again, an objective will want to meet up. Somebody will feel it’s important for the, let’s be really honest, every company there will be certain objectives that feel straightforward and there are other objectives where actually getting together in person might iron out some problems or might feel the right thing culturally for that objective. So we do have some objectives that choose to meet in person as well, which is, you know, it’s a big ask of people to do that as well. You know, there’ll be two types of people and then everybody in between around whether flying halfway across the world for work seems really exciting and thrilling. And there will be people that always feel that and people that are absolutely like, oh no, whatever they need to, they’re a home bird or they’ve got family or whatever, or travel’s not their thing. So we want to make sure when we do bring people together, it’s really meaningful and it resonates with people.Beah: Yeah. Yeah, that’s very much on my mind. In the past couple weeks, I’ve been working with the product team to pick a location for our upcoming meetup. And there’s definitely a lot of people who are torn between wanting to go somewhere exciting and new, but also do I actually want to arrive at this thing coming off a red eye with however many hours on a plane versus a more convenient location that’s maybe a little bit less exciting. So tell me a little bit about like how we generally outside of that anecdote pick locations and dates for these in-person events or meetups.John: Yeah, the dates is relatively simple. We have a, as I say, a cadence of, we try, it’s pretty flexible, but we try and have a six month cadence between those functional meetups and the all company meetup, which just feels a nice balance. So our functional teams, we try and get them together sort of March, April, May time. And then we do our all company meetup roughly six months later. And we do that, you know, September, October, November kind of time, roughly. Sure.Beah: Can I interject a detail about the timing of the meetups? So we used to just say, basically pick it. Well, several years ago, we were just like, it’s all on you, functional team. Just do what you want to do in terms of dates. Then we kind of said, try to do it in the spring, that sort of April-ish timeline. And then, and now, I think maybe this year or last year for the first time, we’re like, try to do it in one of these three weeks. And the idea there, I think we’re still, it’s like, you can tell me if I’m wrong, but still open to evolving that model. It’s a little bit tricky when you have two months where there’s always one or two teams missing at a time, so consolidating that. But we also don’t want to force teams to pick a single week that might not work for that team, and so trying to find the right balance there.John: Exactly. Yeah, exactly. What you’ve just said there, I think could be applied to so much stuff at a company like ours that has gone from 30 people to close to 400 people, where you want to keep that magic. You don’t want to become a really processy policy based company. That’s not what we want, but you’re absolutely right. As teams get bigger, we just need a few more guardrails on it and a bit more guidance. That’s what we’ve tried to do over the last couple of years. So you’re absolutely right. With this year, is the first year we picked like three core weeks where we said to teams, if you can go away during those three weeks, brilliant. If you can’t, the main goal is to get people together. So there’s no point going away as a team of 15 people, if only three of you can make one of those weeks, you know, so we will. And that seems to have worked pretty well this year, sorry, there’s one week where we’ve got, you know, close to half of the organization out for a week, which sounds odd, but it means we can kind of shut down for a whole week. And everybody’s aware of that and then start fresh the week after.Beah: Yeah. Yeah, yeah. All right, maybe not shut down if half the company’s available. I do, I do, yeah. Yeah, you rip the bandaid off basically. And I should clarify, at these functional team meetups, the agenda, it’s not work as usual. I think everybody’s using their best judgment and if there’s something that is time sensitive, sure, they might be progressing something or unblocking something, but like the majority of the agenda is about like taking advantage of being together in the room and working through things outside of the normal flow of work.John: No, shut down, clearly. Yeah, but you know what I mean. You can put it into your project planning. Exactly. Yeah. Absolutely, yeah. So it’s not work in its traditional sense. Every team will do what’s right for them, but I think we all work in a similar way, this is how we guide teams to do it. So if I look at my own meetup with my team, we didn’t sit down and do core people-based work, but we did some sessions where we had some special guests and other teams kind of sign in on Zoom while we were all together in the room and we had little chats with other people at DuckDuckGo about AI, for example, and we had someone from the data science team talk to us about evaluation methods, because that’s something we do a lot of. So we will absolutely have some work based chats and we did some personality stuff, which was kind of fun. Yeah, the rest of the time is just hanging out and it’s pretty unstructured hanging out. We do give some guidance on what kind of works well for teams, especially bigger teams. It probably goes without saying with our company, but we’re not, you know, unless the team wants to, we’re not going to a riverbank and building a raft. It’s not, and doing trust falls. It’s not that kind of, that’s not what our meetups are about. We want people to just organically hang out. So, you know, we do do activities. Actually that does sound all right. That does. It’s yeah, but we definitely want the teams to have the choice of what they do essentially and what works well for them. So.Beah: Going to a riverbank and building a raft sounds great. But okay, that’s just me. That’s just me. Yeah. Yeah.John: So yeah, so that’s the content on them. And go back to your original question. Sorry, go on, go on.Beah: And, and no, well, so in terms of like the whole company meetups, I think kind of the same principles apply, right? Like we are, it’s not like, it’s not trust falls unless you really want to do trust falls in which case you can do trust falls. You and I are definitely doing trust falls. I feel like this whole conversation is a trust fall. And it’s not, but it’s also not really like decision-making work, right? I mean, I’m thinking now about the whole company meetup. We’re not coming together in the annual, it’s not like an annual meeting of business where we’re trying to resolve things. I mean, you talked about just relationship building is a big part of it, getting to know each other. And then I think, but you tell me how you think of it. It’s like the other big component is kind of basically learning, like context building and learning from each other. No, you tell me the correct answer.John: No. Exactly that. No, exactly that. There’s probably a main aim for it and then some sub aims that we have for it and objectives. The main aim is, our chief exec Gabriel will talk about this when he stands up at the start of the meeting, it’s to hang out. It’s to spend some time together. And if you look at it in business terms or psychology terms, that’s about trust building, that’s about collaboration. But essentially it’s hanging out. It’s getting to know people face to face. And we really evaluate our meetups, but there’s a certain amount of sort of unmeasurable magic that comes with that, that just feels nice about being at those meetups. So yeah, we do some work sessions in the morning. As you say, explore each other’s work that we’re doing. We do some pretty informal sessions, presentations, and a lot of the time when I say presentations, it may be just a couple of slides and then a chat about an objective and people can go and learn about what everybody else is working on.Beah: Okay.John: Even those can feel pretty chill. And I can tell we do them in a chill way when I go and speak to the venues. You know, we might book out seven rooms at a venue and we’ll have concurrent sessions going on. And the venues obviously work with companies where they’re very, how many chairs do you need in this room? And what do you want us to do with the empty chairs? And is everybody mic’d up? And do you want us to, and it’s like, it’s just cool. If the mic doesn’t work, we’ll all just have a chat. And if there’s not enough seats, I’ve seen people sitting on the floor and that’s not ideal, but it’s fine. We’ll be all right. We’ll be cool. So even that has a chilled feel. And then, yeah, we spend the rest of the time at our all company meetups doing fun stuff really. And go back to the point of, none of that is mandated. So if you’re somebody that finds it a lot and it is a lot coming away for a week and being with 300 colleagues suddenly, it’s a weird thing. It’s a lovely thing, but it’s a weird thing and it can be tiring. So we don’t mandate anybody doing anything really. We want everybody to come along to the work sessions because that’s a really nice opportunity to hear about what the organization is doing. So that’s as close as we get to mandating something. But in the afternoon, and this is what I love about DuckDuckGo, it’s a lot of self-run things that people at DuckDuckGo are passionate about. So we’ve got a lot of people that are passionate about board games. So I don’t arrange any of that. I literally find a space and someone at DuckDuckGo will set up a board game room. You know, we have a chess tournament, we have art classes, we’ve had writing classes in the past, we do a lot of sporty stuff. You know, people that are passionate about basketball or softball, actually people that aren’t passionate about those will go along and just hang out and have fun at those. So yeah, it’s a very chill environment, a very, and that seems to work. It seems really nice. Yeah.Beah: Yeah, I love personally that I do all the sporting stuff even though I’m not a very skilled athlete. I would say I’m passionate but not skilled. Like the only time I play basketball, soccer slash football, softball is once a year at the company meetup. But I love it and it’s so cool because like I actually, I didn’t play, this was the first year that I played soccer slash football. And I’ve never in my life been a soccer player. And there’s people at DuckDuckGo who have played borderline professional football in Europe. Like, people who have actually played on teams where people are getting paid to play really well. And then there’s me who did not even play in a kids league in the US. And somehow it is just magical to like see how the teams come together and just the level of camaraderie and respect and everybody kind of meeting each other where they’re at. I was genuinely a little nervous that it was going to be like, that I was going to be, yeah, or that people would kind of feel like it would be annoying to have me on the team. Like, well, maybe it was, but nobody acted like it was. So I had a great time. I had a great time. Yeah.John: Yeah, super competitive and... I didn’t hear anybody saying that. There wasn’t any, yeah, it’s cool. I think for people, you know, there might be a lot of people listening to this going, how cool does that sound like going away? And there is, I always think my job is one of the most fun jobs in the world. Like I’ve got such a cool job. I get to go out and sort all these fun events. You know, I’m not sorting an Uber like to the minute work event where I’m very stressed about this person coming on stage at the right time. So we’ve not got that, which is lovely for me. And then the stuff I do arrange is things like, yeah, I’ll book a softball field for people. And it’s such a lovely job to be able to do that. But I do always say to people, like, if I was doing that for entitled people, my job would pretty soon be terrible. Like if I had a bunch of entitled people turn up, because, you know, things go wrong at these things, or, you know, people are tired from travel and that’s for me, what is lovely about our meetups. It’s full of people doing lovely stuff for the other people at DuckDuckGo. Like if they’ve got a passion, as I say, like, you know, I’ll get a ping, hey John, I’m really passionate about this role playing game that we do. Could we do it in the meetup? Of course, yeah, that’d be great. So it’s self-organized and you just get happy people doing fun stuff and it’s lovely. Yeah, a bunch of unentitled people just enjoying themselves. It’s nice, yeah.Beah: Yeah. Yeah. Yeah, I know now we’re just in the realm of like, we’re so great. Such a lovely group. Tell me like, before we wrap this up, I think this podcast has gone longer than most because I mean, really, what were they thinking when they put the two of us in a room? Were they thinking that it was going to be 12 minutes?John: It’s so nice. Yeah, it’s so lovely. Yeah, sorry, that’s me. I know and it’s fun to talk about travel rather than, you know, email security or whatever, it’s a lot of fun of course but I’m just saying yeah sorry cool.Beah: Yeah, it is fun. So tell me though, are there any like, is there like a magical meetup moment that we haven’t already mentioned that you want to cite?John: I guess I’m so cliché, but I think most of the magical moments are kind of the smaller ones really. You know, from people doing events, like I, I’m normally running around doing stuff, but the year before last, I did get to play like soccer and stuff. So I agree with you. Like that sort of stuff is kind of feels fairly magical for me. It’s like the smaller stuff. So, you know, when I, to sound like a proud parent or something. When I wander through the hallway and you maybe see, I don’t know, our CTO who’s been here since almost day one sitting with a new starter who’s been here a month from marketing and then maybe an engineer in that team and you see them all just having a coffee together. That’s kind of a magical moment because that doesn’t, we do engineer that to happen a bit virtually but they still, I don’t care what anyone says, there’s still that slight cringey, slightly awkward thing about virtual socializing. There is, isn’t there? We remember it like during COVID. Like it was fun and we had to do it and we did Zoom calls with the family and stuff, but it never feels quite as organic as like real life. That for me always makes me smile at the meetups when you see a group of unexpected people together doing things. That feels like a sort of a magical moment. I guess for me as well, I would say like we’ve evolved it the last couple of years, but our welcome session always feels really magical. You know, this year was the first time we did that in the evening. And it was a bit of a celebratory thing. So I know you were on stage sort of talking about product stuff and we had a number of people that felt really cool in terms of a celebratory element of where we’d got to over the year and an update of where we were at. But still a good laugh, you know, and people really, you know, getting involved with it with the vibe of it and stuff. So, yeah, I would say those are the magical moments. I guess if people are like hearing that and going, he sounds a bit corporate saying that. I’ll be really honest, like, we’ve stayed in some like beautiful properties as well, like not necessarily like really posh, but like in beautiful places. So you do, I’m sure the vast majority of people who work here, sometimes when they’re getting a coach in the airport and it sort of takes that last turn and you’ve got like a mountain range, for example, and like this old hotel that you know is all of ours and there’s a lake and it’s like, wow. So that I think for a lot of team members that feels a bit of a pinch me kind of magical moment I think that always feels nice yeah.Beah: Yeah. Awesome. All right. Well, I think we’ve officially deeply exceeded the time limit. So I think we should wrap John. Yeah. I think so. I think people, I think people would listen to that.John: Yeah. Okay, let’s do that. We’ll do like a three hour, we can do a three hour version, the two of us. Yeah, like we can do a long form. For our subscribers, that’s what podcasts do, isn’t it? We have a short, okay, as long as we get him or her as an audience. Yeah, that’d be great. Yeah, no. Thanks, Beah. It’s been fun. Yeah, cool. Yeah, you too. Thank you. Cheers.Beah: My dog would. Yeah. Yeah. Okay. Well, thank you so much. It was nice. Yeah, nice chatting. All right. Later.Sonnet 4.6Claude is AI and can make mistakes. Please double-check responses. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit insideduckduckgo.substack.com

In this episode, Mary (Senior Director, Brand), and Nirzar (AI Design Lead) discuss why we developed a new typeface, how we implemented it, and its role in communicating our personality. Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.Mary: Hi everyone and welcome to DuckTales where we go behind the scenes at DuckDuckGo and discuss the stories, technology and people building privacy tools for everyone. I’m Mary. I work on the brand side here at DuckDuckGo. Today we’re talking all things typography and more specifically how we developed DuckSans, our new custom typeface. So I’m here with Nirzar who beyond being our product design lead for AI, he’s a real typeface nerd and complained about our previous default typeface for, I don’t know, like five years, five plus years. So we had him lead the effort so he will be able to answer all of our questions. Nirzar, thanks for joining.Nirzar: But yeah, Hey, hi, I’m glad we’re talking about fonts.Mary: First prop use of the conversation. All right, I’m jumping right in. For folks who are less familiar, why does a typeface matter? Why not just use the defaults? What is the benefit? What are you trying to do by creating a custom one?Nirzar: Yeah, that’s a good question. I think there is nothing wrong with using defaults to be honest. Nowadays default typefaces for operating systems are very well designed. Take San Francisco, Sans for example, by Apple is very well designed typeface and it’s going to be fine. I think the main part of this is mostly about what our product is. DuckDuckGo is a search engine, have UI chat. Most of our product, although it seems like it buttons in Chrome, Windows, most of it is actually with text content, like we take search results or take the AI chat with Apple. So around like 80% of product surface that you’re seeing is typography. We spent so much time on like thinking about color, this, that, buttons, styles. I think typography and type setting kind of require that much attention as well because they take like most of your screen when you’re using it. The reason for not using defaults, I think it’s where kind of the brand maturity sort of goes into it’s something we wanna do to kind of associate a kind of a feeling, kind of a look that we want to encode in our brand. I mean, I can go into a lot more detail about this, but just the idea is like, you want to make... Yeah, but like, yeah, the idea is you’re basically kind of creating an ownable sort of like element. I think typography is as important as the blue color that you use and the brand colors that you use and everything else.Mary: I know you could. I know you could. I think the search engine as a surface is a good point to bring up because, know, obviously, like I mentioned, I work on the brand here at DuckDuckGo. And when you look at the search engine, you’re often questioning, you know, how can I inject more personality? How can I inject more of our, you know, our over our affect into the product and make people know when they look at it that this is DuckDuckGo. This isn’t Google. This isn’t Bing. And there’s really not much you have to work with.Nirzar: Right.Mary: Obviously you have the logo, but the typeface makes a huge difference. And so, for folks who haven’t seen it, this is where we rolled it out first. So you might have noticed a difference, but this was kind of one of our key areas we were most focused on. Nirzar, we began the exploration, what were the factors you were considering? Because obviously there’s hundreds of thousands, if not millions of typefaces that have been customized to choose from. What were you looking for specifically?Nirzar: Yeah. I think the process starts with just collecting what are the use cases that we have, obviously, and what are the goals we have. I think in speaking about the goals, I think the main and most important thing, goes without saying, is just legibility and readability of things. And when I say that, it’s a little bit different than designing type setting for a book or something where it’s going to be read in a very specific setting and control that, you know, the paper you’re going to print it on. Like for us, we are talking about across devices, across platforms, across different types of screen densities. Like there’s many, factors that come into play when like somebody is going to look at your work, the design work that we’re doing or the product and run work we’re doing. So I think just considering all of those, tying that with this making it like the most functional but at the same time having more character so there are these like opposing sort of challenges as well you can’t have a lot of personality because then readability suffers if you focus only on readability then personality suffers then you have something that is like that looks bland and default it is extremely readable but you can’t tell it is DuckDuckGo so we were kind of like talking about the challenges and the spectrum of where to land on personality versus readability. And yeah, I think that being the goal, I think that’s where we started our exploration at.Mary: Mm-hmm. You. Yeah. Was there something about the DuckDuckGo personality you wanted to bring out in particular? I mean, as folks know, we have a duck for the logo. Like there is a whimsical element to the brand that but you don’t want to go as far as like comic sans by any means. Like what were you most looking to play up when you talk personality? What was your goal?Nirzar: Thank. Yeah. Yeah, I think it’s a very difficult challenge. I mean, definitely our brand is kind of, I don’t know, I really like it. It’s very quirky and sometimes goofy and derpy as well. But at the same time, we also care about trust and other things which are kind of, can seem very opposite point of view. But for me, it was more about like bringing a little bit of sort of pluck to it. So, and like the way it kind of gets codified into the shapes of the letters is more about like how certain things, what is the angle at which you cut the corners on a terminal of C for example, or the way you look at DuckSans Q, it’s very sharp and it’s very straight, but then you and other letters kind of complement the roundedness and friendliness into it. So it kind of like the, the kind of characters you’re looking for kind of trickle down into these like very specific things about shapes. Yeah, I mean, it’s not perceptible like right away when you look at it, but it is thought out. There’s thought behind all of these things.Mary: Yeah. Mm-hmm. Hmm. That makes sense, yeah. But there is, yeah, definitely. I mean, even thinking about when you talk about it being standardized versus like when you change certain shapes, you’re making it less uniform in some ways. You’re making them certain things stand out. Like we spent a lot of time, I remember deciding how big to make the dot on the eye. And we were like, no, like a little bigger actually. And so it’s like to your point about it not being immediately perceptible. I think when you take a step back and you see it all together, maybe you notice something that doesn’t look quite as uniform, quite as standard.Nirzar: Yes.Mary: Which is what the brand is going for. But it is funny when you end up fighting or not fighting, discussing the I dot on the letter.Nirzar: Yeah, I mean, it might, like, I really think, I think if you don’t work in typography or in design in general or brand, you might think that we’re just like fighting over or discussing non consequential things. But actually, what I care about mostly is having meaning to it and not just doing it for the sake of doing it, but actually like putting meaning behind, codifying values and trust and all of these things into visual like artifacts.Mary: Yeah. Yeah. Yeah. Yeah, so let’s get into a little bit how we found and started creating DuckSans. So DuckSans is based off of Pangea. Do you remember how you found Pangea or how you came across it?Nirzar: I don’t quite remember but so Pangea is a typeface that was designed by Fontwork. It’s a type foundry in Germany. I remember using a Fontwork typeface few years ago on a project. So I basically like I have a that’s my like thing that I do in free time go to type foundries and look at that.Mary: Yeah. Don’t tell people this. It’s too revealing.Nirzar: But yeah, I think there has been a renaissance in type foundries recently. I think there’s a huge amount of work that is happening. And there’s a third wave of typography coming in, and digital typography particularly. There’s a lot of experimentation going on, variable typefaces, or even just doing very whack things, which are very cool in the last four or five years. So I was very excited to take on a typography project. I was like going through the Rolodex that I have for all the type boundaries to look at what is happening everywhere. Yeah, I think that’s, but I think particularly Pangea caught my eye just because how versatile it looked on the surface. It’s a variable typeface, but it’s just designed for scalability. And it also had a little bit of character to begin with before we customized it as well. But I think those were the kind of two things we are looking for is like something that is like durable, scalable and something that has character at the same time.Mary: Mmm. Yeah. Did you feel that Pangea, sort of as a base, represented a lot of that legibility concerns? So we customized Pangea, so that’s where we ended up with DuckSans. There’s customizations and things we built in. What specifically, from your perspective, were we trying to achieve with the customization? Is it bringing out more character, or do you see it more as an engineering changes?Nirzar: Yeah, we can get to engineering in just a bit. But before that, I think when I say scalability, I mean like literally scalability. We have a eight point gray link somewhere in the footer and we have large billboards. So like it’s really difficult to find like a silver bullet solution that can fit both of these cases. So I Pangea scales really well in terms of it’s actually two different typefaces, is specifically designed for to be used on large scale. This is what we call display typefaces. And we have a product layer which is specifically designed for legibility, readability, screen rendering. It can be readable on like 8.6 GB or anything like that. It’s just like made specifically for that. So this is what I mean by those two things. So that’s like part of, a little bit part of the customization. The font engineering is like a lot more deeper.Mary: Hmm. Yeah. Yeah, before we get into that, we, know, like to your point of like these nuances are things we notice, but not everyone would like we wouldn’t we one of the things we did was we tested this. So we did some pre testing with users or we’d show them our previous default and this one in different contexts. And I will say I was really surprised by the overwhelming preference for this new font. I was curious, like, were you surprised and what can you what can you share about the testing?Nirzar: I was definitely surprised seeing a statistically significant difference between to grow test typefaces that are very similar. I mean, we tested what, like 2050 typefaces. But I think seeing that much difference in like perception was very, actually very comforting on the project itself and also like for me. Because I think it’s something that you are like so akin to, like if you use the product a lot and like you’re not gonna see it in one second like it’s it’s a longitudinal thing if you’re using something for 10 years and like suddenly the dot on the eye changes you’re going to notice it like you’re immediately going to notice it the last typography work that I did was with Wikipedia and I was like very contentious because there’s such a huge sort of like your eyes are kind of like dead set on like how it should look and if you think that it’s very very apparent exchange. But capturing that in the setting that we did was actually a surprise because you’re kind of showing a little bit of the context because of the constraints we had. It’s not something that you want to use and watch somebody use, get their feedback on these things. But yeah, mean, maybe people are a little bit more sensitive to these things right now than before. So I think that’s what kind of points it to me.Mary: Yeah. Hmm. Yeah, I would believe that. And I mean, we tested it across like read, like, you know, you’re more functional, like readability, clarity, but also like, which is more memorable, which even just which do you prefer? Which do you which has more personality and overall, you know, to your point, like you could argue that two grotesque fonts, they look very similar, but people really did have a preference. And I think we’re really happy to see that it aligned with a lot of what we also felt was stronger and better represented the brand.Nirzar: Yeah, I think that helped us a lot moving forward because I think it’s very difficult to kind of have similar things that both have trade-offs. I think research really helped us.Mary: It. So what have I not asked you about the design of the typeface that you want to share? Because I’m sure there’s things I have not asked you. Because I want to ask you about the engineering effort. Because obviously from our standpoint, there are a lot of improvements we made to the readability of the typeface. But what have I not asked that I should?Nirzar: I don’t know, I think... It’s really important to kind of like, like typefaces have become important. I think a lot of companies like Facebook and Google and everyone started investing into typography very recently. And there’s like wave of thing about like, everything started looking very similar to each other because it was all like indexed on how readable it is. And you can make an argument being like, the logical conclusion to a modern typeface is what everything should look like. I think that’s something that I struggled with personally as well, is to kind of like, how do we kind of break apart from it, but how do we also serve the function that typography is supposed to do? But I don’t know, I think it’s a spectrum. I think you and I worked on the spectrum a lot to figure out where we should land.Mary: Mm. Yeah. Rest in peace, Rubear.Nirzar: But yeah, Ruben is good. But yeah, I don’t know. It’s not a question that you didn’t ask. But I think it’s something that I think it’s worth calling out in this conversation, being like, hey, did you just make something that seems like everything else that is happening? And I think it’s the same with all logo as well. It’s like we can’t make our duck very much look like any other software company logo, but I love that we kind of keep that quirkiness and like deal with all its quirks like on everyday life when we are working with it.Mary: I know he can only look right. He only has one. He can only see one direction. In terms of where we go from here for the typeface, you know, marketing, you know, one use case we’ve talked about is like where you can bring out more personality and characters when you’re doing marketing, advertising, things like that. Like, I think what we focus on was the product use case. But I imagine that there’s a lot of future work still to come when it comes to bringing like even amplifying that character more. Is there anything you’re particularly excited about or is it just a bigger effort to undertake that is upcoming?Nirzar: Exactly. I think it’s more about having the capacity to, the capacity and foundation in the typeface to be something different for different scales and purposes. And that’s what we kind of like, I think we achieved that with DuckSans because like, if it’s not in a experimental enough, like identifiable, I think we have a great foundation in the structure of typeface to kind of like drive that chain forward. Like we have to keep those options open. So I think this extensibility, this property of extensibility is really important. And I think, yeah, I’m like super excited to like tomorrow figure out like, maybe we can do something weird with a of letters. And when it is used, I like 200 point size on a billboard and I think that can still carry the essence of main typeface but as a version that is made for particular use cases. I think just having that optionality itself is kind of like really good.Mary: Hmm. Yeah, that’s a great point. I think my favorite is we have a normal ampersand and then we have an ampersand that looks like a duck, which we have yet to use. So you’re hearing it here first. Easter egg. It’s an Easter egg that has yet to be put anywhere. I don’t know. All right. Let’s talk a little bit about engineering. Let’s do Mohammed Proud here. Can you talk a little bit about what the engineering effort was like to get this on the SERP. Like I remember hearing about these like hour long calls where in my imagination you were like, Mohammed, add a space here. Like it was like, I was imagining you both like sort of custom fiddling and like changing the font manually. Was that what was happening or what was the process of typesetting this font?Nirzar: Yes, it’s an Easter egg, I think if you find it. Yeah. Yeah, so the type setting part is actually took the longest time as well. I mean, like, yeah, you’re right. We were on calls for hours kind of figuring out how to implement and how to make use of it. The things that we were kind of working on an improvements from the status quo where things like using new rendering methods on web, something called like anti-aliasing and corners, painting on windows, our previous life was particularly broken on windows. I was very, whenever there was a curve anywhere, pixel part. But we were kind of like figuring out what are the best methods and the most modern methods to render on screens. The other part that we obviously have to think about, we are a website and an app at scale. Performance is key important metric for us. So just thinking about how we can reduce the load time, we are basically kind of improving upon a typeface, but new features mean more footprint. We definitely don’t want to delay page rendering in any way. If not, we have to make it better because we are now looking at this part of the software. So we’ve been working on optimizing how we can optimize the delivery of font files. So that goes into just insane amount of splitting our Unicode ranges from WAF2 files into this file and that file, basically having particular letters, let’s say your page is Greek, we just want to load the extra letters from the other character set into the main one, and not actually the separate typeface that has Greek in it. So just optimizing, optimizing this to the point of like just being like very bespoke about all of this because again, it’s like millions of users like everyone loads this all the time. So like being really on point and really getting into like the nitty gritty. Like I remember like working with Christoph from Fontwork where we are trying to separate characters for Cyrillic, but we even went further and we want to load only the accents not even the actual letter because we are like the a is the same but the accents are different so like let’s even split that out and there are these a lot of font engineering techniques like font work helps us in optimizing.Mary: Yeah. Mm-hmm. Hmm. Is this like, is this trial and error that you’re finding this out or are you deep in some Reddit threads on type implementation?Nirzar: Actually, yeah, I would have thought it would have been easy to find guidance on, this is, we are talking very specific things for very specific software. I didn’t find, because most people would give up on level two of optimization. We kind of went a little bit overboard. Actually, we went overboard that after we figured out our page, our search results are going to come after the point at this point. So, we’re like, we like this.Mary: Hmm. Yeah. You. That’s amazing. Okay, I’m taking this completely out of this project. How many fonts do you think you have downloaded on your computer? Like, not even related to this, just generally, like how many custom fonts exist in your device?Nirzar: I don’t know. Thousand? I want to say more than thousand? Hundred percent, yeah.Mary: That was, that is insane. All right, any final words from you Nirzar on typeface, DuckSans, anything?Nirzar: No, I mean try it out give us thought I was very happy like some of the community members noticed and actually said nice things about change which is a bit rare on the internet but it’s always good to see that. But yeah it’s like I’m like more excited to kind of like now propagate this more in other parts of the brand the word mark and things as well in the future. Also my typography professor would be proud.Mary: Very proud. All right. Do you want to do you want to finish off by plugging your new album or should we leave that? All right. Thanks, everybody, for listening. Tune in next week. Yeah.Nirzar: Oh my god, who’s that? Hey. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit insideduckduckgo.substack.com

Curious how people voted? Head to VoteYesOrNoAI.com Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.Kamyl: Hello and welcome to DuckTales where we go behind the scenes at DuckDuckGo and discuss the stories, technology and people that help build privacy tools for everyone. In each episode you’ll hear from the employees about our product vision, product updates, engineering, approach to AI, et cetera. I’m Kamyl Bazbaz, SVP for communications and policy at DuckDuckGo. Today I’m with Mary McGee, one of our senior brand directors and I’m very excited to talk about something that we worked on together, which is our Yes AI No AI campaign that just launched. Mary, do you want to introduce yourself and then we’ll start talking about the campaign.Mary: Yeah. Hi, Kamyl. Great intro. So good. I, like Kamyl said, I work on brand here at DuckDuckGo. I’ve been here about six years and I’ve worked on everything from our homepage to onboarding to messaging, all basically, how does it feel to use DuckDuckGo? How does it feel to use the product? What are we trying to communicate to you, our users and the types of conversations we’re looking to have? So I’m excited to talk about this one. It’s a sort of a new effort on our side, which hopefully we’ll see a lot more of.Kamyl: That’s right. So I guess let’s let’s start from the basics. What is the no yes campaign and what are we asking people to do?Mary: Yep. So this is our live public vote on AI and where it came from. Actually, it started probably back over the summer. So, you know, a lot of users will know that we’ve had a version of our search engine, noai.duckduckgo.com for maybe the past six months. And we built it as we were building our AI tools because something that was really important to us was that AI be optional. And that actually, comes from this sort of long history of DuckDuckGo really prioritizing user choice and wanting to let users decide how and when and how much, you know, you’ll notice that throughout all of our products. But when it came to NoAI, we noticed that there was something to this. There was, was, you know, it was getting used, people were talking about it. And it kind of sparked this conversation of like, you know, we believe optionality is, you know, inherent in good technology and letting users make that choice. But we noticed with AI, people really aren’t being given that choice. It’s like every time you see, you got a new email about a new feature or a new product, they talk about their new AI integration. And there’s no talk about how to turn it off. There’s no talk about, you were asking for it, here it is. There’s this sort of like, there’s this real gap between user demand and what companies were releasing. And so, like I mentioned, when we were thinking about AI, we were thinking about useful, private, optional, and there’s something about optional that really stuck out. So that’s where this campaign came from. What we wanted to do was actually ask, a thing that tech companies don’t often do, actually ask people, hey, where do you stand on AI? What is your take? What do you want? And we asked in really a simple way because for us, we sort of can help you either way. If you’re yes AI and you’re like looking to try AI chats or AI summaries, we can help you. We have a private version of ChatGPT. We have search summaries. That’s the yes AI experience. But if you’re no AI and you just feel like AI is being built for someone else’s vision of the future and you’re not represented in these products, we have a no AI version where we take it out and we even go further and remove AI imagery. And so really where this came from was we feel like optionality and choice matter. And so we wanted to create a public vote to help make the case that so do you, so does everyone. Like fundamentally, even if you love AI, you probably want the choice over when and how you use it. And so the idea is to get people to vote, show where they stand, explain why, and all resulting in the idea that like this should be optional. People should have the choice. And that’s what we’re hoping to find and create more of a conversation about.Kamyl: One of my favorite things about the campaign are the updated landing pages that we made on yesai.duckduckgo.com and noai.duckduckgo.com with decks in the middle and a very clear no or yes AI. And so I’m wondering how that came together and what was the creative development around that.Mary: Yeah, the it was actually the first thing we did like before we even decided on the vote or the billboards or anything else we’ll talk about today. It started with these pages because we fundamentally were like, this is the point. This is giving users choices is the representation of it. And so the idea was like, how can you create it a very simple experience? That’s very clear. I think there’s a version of this where you just disable the AI features or enable them and leave it at that. But we wanted these. We wanted to really reassure people about which experience they’re in and make that really clear. So that’s where, you know, just really leaning on the no and yes and the big bold, making sure you knew which experience you were in was really important. And I wouldn’t be surprised if this is something we like continue to think through and find the best way to make sure we’re like serving both sides of this conversation.Kamyl: Yeah, I think you sort of touched on this already, but I think one of the things that I really liked about this campaign from the start is that it felt like something only DuckDuckGo could do. We pride ourselves on optionality. It’s sort of part of our legacy. You should be able to choose privacy. You should be able to choose the default search engine. That’s been a cornerstone of our advocacy for a long time. And as we know, Google, for example, doesn’t give their users an easy way to turn off AI if there’s really an option at all. And so this felt like something that DuckDuckGo had a really unique and credible position to take.Mary: Yeah, absolutely. And I think, you know, speaking specifically of the kind of like no AI side of this, there are many people who want to opt out of these features, you know, and they are really not being given this option. We really felt like people deserve that level of choice. And it just feels like there’s this big group of people just getting ignored right now. And so some of this was both to acknowledge that and make sure they felt like they had, they could use the internet like they want and also create a conversation online and space for them because it’s nobody’s sort of mark talking to this group. You know, some are, but just not quite in this way.Kamyl: So this, you know, people will watch this interview next week. The campaign will have concluded by then. Right now, it’s Wednesday, January 21st. We have over 85,000 votes. And if you want to talk about what the split has been or not split and sort of your reaction to that.Mary: Yeah, so even from the beginning, it’s been pretty much like, I think since the beginning, it’s been like a 93% no AI to 7% yes AI. And it’s really maintained. Like we’ve been watching these graphs from the beginning just stay perfectly in parallel, which has been really fascinating. And I it’s really exciting. Like, you I think there’s been comments on social that’s like, is this what DuckDuckGo expected? You know, all we wanted was to create this conversation about what people want and what they expect from technology and what they’re asking for. Like, we don’t have an agenda. Like, we support optionality, which means we support creating experiences for what people want and serving them both. Really, there’s not, it’s really not about either side winning. That was never sort of the point. It was about creating a conversation about what people want and giving them this like opportunity to express it. The question is really simple, yes or no. And obviously the answers aren’t simple. You know, sometimes people are yes for some things and no for other things, but that’s what we were hoping people would talk about on social. Like take your yes and say what it meant to you. Or, you know, like I’m no, but yes in these ways. We have some creators participating as well. And that’s a lot of what they’re bringing to the conversation is like the nuance, the context, but for us, is like a, we’re really excited to see the conversations people have had. And this is just like exactly what we were hoping for is to create this like very clear stance that like what people are being provided is not what they want. And that fundamentally like from our perspective, that’s why we’re where we are promoting optionality. That’s why we believe that’s important.Kamyl: I think it’s fascinating that we’re a company that gets to live inside of this sort of tension and conflict that, you know, we can spend so much time making noai.duckduckgo.com and have people get extremely excited about it and feel like someone is finally picking up for their point of view. While at the same time, we also have our own internal data from surveys and other research that you know, our AI assist feature, which, you know, uses AI to give, to really expand the number of instant answers, what folks on Google would call the knowledge panel answers, knowledge graph answers, expand those numbers significantly and makes the search experience better. So I think it’s so interesting that we can sort of do both, you know, use AI to improve our services and get good feedback on it and also give people the option to just not have it at all if that’s what they want.Mary: It’s just so interesting that like, it’s that it’s revolutionary, you know, like, or that it’s so rare. Like, it feels like the most simple thing in the world to be like, here’s a new feature. If you don’t like it, you can turn it off. You know, we, you know, that’s been Gabriel’s, you know, our CEO and founders, like method from the beginning. Anytime we release something, he’ll ask, where is the off switch? You know, it’s like, how can we give that control? It’s just so strange in the era of like, all the algorithms and personalization, it’s like you used to be able to customize everything and now they’ve sort of assumed what the best experience was or taken everyone to the mean. And it’s funny to think of this as different when it so used to be the norm and we’re sort of maintaining that.Kamyl: As this is DuckTales, and we have to go behind the curtain a little bit, one funny thing happened where noai.duckduckgo.com existed prior to this campaign. We just sort of updated it for the campaign itself. And it was discovered that we were having this vote before we even launched it this past Friday. You want to talk about that a little bit and what happens?Mary: Yeah, well, you laid out what happened exactly. We ended up redesigning what noai.duckduckgo.com looked like, and there was a little Easter egg. Like if you hovered over the big logo, you could see a link to the vote. And so we ended up pushing that live to run some tests on what it looks like when you share on social, things like that. And it ended up getting discovered and shared on Blue Sky. Within, I think a, an hour or two, we had 10,000 votes, we had people engaging about it, lots of excitement and questions. What was really interesting about it is like, how can something like this live on its own? We had prepared social posts and Gabriel, our founder wrote a post about what this meant to us. But when you take back how we were framing it and you just brought this debate to people, which is what happened when it leaked. How do they interpret it? What do they do with it? How does it create conversation? What was really nice to see is the optionality point really did come through. I think it got, this question of yes or no, it being too simple, but understanding the different contexts as you want it and don’t. In many ways, I wouldn’t change how it happened because it was great to see it sort of embraced and like championed by the people we were trying to have the conversation for, and then it let us join the conversation actually in a funny way when we officially launched on Monday and bring it extended to other channels.Kamyl: Yeah, I think by Monday, we basically had almost 50,000 votes with zero promotion on our part, which is not something we expected, but I think, yeah, it’s that we start the court.Mary: Yeah, and it’s this conversation wanting to be happening or that is in many cases was already happening, but was creating like a different way of talking about it, accessing it.Kamyl: Yeah, great. So before we wrap up our conversation here, I think we’ve covered everything. Just quick wrap-up on our questions for you, since again, we’re trying to pull back the curtain here. You’ve already said how long you worked here, six years. That’s many years in duck years. Why do you work here?Mary: Where is this curtain? Where does the curtain exist?Kamyl: You know, it’s private, except at the elementary, so it’s none of your business. So you already said how long you worked here, six years. That’s many years in duck years. Why do you work here?Mary: Yeah, none of my business, none of my business, yeah, okay, yeah. I, well, it, know, this is an interesting story to share with you, Kamyl, because Kamyl and I actually worked together before DuckDuckGo and we worked at a consultancy. And so we worked with a lot of different, mainly tech clients across, you know, a lot of industries. And what I loved about DuckDuckGo was like, felt like you could be, actually we’re trying to make the internet better. Like it was a tech company that really was mission driven, but was also made, you know, researched back data decisions, you know, lot of the companies you see or products you see released are sort of, they don’t have long last, there’s not this like long lasting legacy to them. Like DuckDuckGo has been around since 2008 trying to do the same thing, you know, giving people who want more privacy a private real alternative. And as the internet changes, it’s new products, it’s AI or it’s browser or it’s app tracking protection, whatever it may be. It’s sort of stayed the course and remained that like steadfast company trying to do the right thing. And it just attracts a lot of people that are wanting to do the right thing and making it better. And that does, you just like, you know, we consult, I consulted with like dozens of companies and you just don’t. It is rarer than you think. And so that’s why I came and it’s what keeps me here is feeling like you’re still trying to make something better and that has a history of trying to make something better.Kamyl: What’s something that surprises you about working here that people don’t know?Mary: Hmm. I think the company is funnier than it seems sometimes. Know, I think the, obviously have a duck logo, Dax, but a lot of what you’ve seen coming from us has been a lot more about like privacy education or things like that. And so what we’re trying to do over, you know, is, what I’m really trying to do is like sort of unleash the like personality that is so real about the people that work here. Everyone is like, it’s such a good sense of humor. The company is quirky. It’s like full of people that have this sort of like quirky fun personality. And so showing that more and making that more of what your, like the user’s experience is with us. It might surprise you to hear that. And I think we’ll hopefully see a lot more of it.Kamyl: Thank you, Mary. That was really great. So where can people go to check out this campaign before we wrap up?Mary: Yes. Okay, you can do the link up here. I’m just kidding. You can go to voteyesornoai.com or check out any of our social feeds. We have links posted of where you can go.Kamyl: Thank you, Mary. Thanks for everyone for tuning in and reading DuckTales. See you next time.Mary: Thanks, see ya, bye Kamyl. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit insideduckduckgo.substack.com

In this episode, Gabriel (Founder) and Ewa (Product) discuss Search Assist, why we’re so focused on letting users control their experience, and future improvements. Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.Show notes: Learn more about the “More” button in Search Assist here. Gabriel Hello everybody, welcome to DuckTales. I’m Gabriel, founder of DuckDuckGo. With me is Ewa. You wanna introduce yourself, Ewa?Ewa Hi everyone, I’m Ewa Sobula. I’m a product person at DuckDuckGo based in Poland.Gabriel Cool, we are going to talk about search assist today. We’ve done one episode before when we introduced the more button. This is the stuff on the top of search results where our anonymous AI is answering queries for you and you can click more, but we’re gonna go take a step back and just kind of talk about the feature in general. ⁓ As a precursor, I’ve said this a bunch of times. at AI episodes, but ⁓ our guiding principles for AI features are that they’re useful, private, and optional. in this case, and I know people really appreciate that, ⁓ we think Search Assist is extremely useful, and we’ll get into that, but it is also optional. So you can turn it off if you like. It’s really easy to do so. There’s toggles actually within Search Assist itself, but also in the browser and search settings. And of course, it’s private like the rest of our search results. It’s completely anonymous. ⁓ So with that, yeah, let’s just jump from the highest level. What is Search Assist and how does it work?Ewa So as you mentioned, search assist is our AI generated instant answer to search queries that we show up on top of search results page. Once we are confident that this is gonna be the optimal answer to use a query, meaning for queries where you either ask a question or really are looking for a quick summary. And ⁓ we are now showing it on like roughly quarter of our searches and we are using LLMs to create the answer but what is important is that we actually grounded in the right sources and like verified and checked sources so it’s not like generated literally and just by an LLM but we find the relevant sources to the search query. we analyze them and we synthesize the concise one, two sentences also that we show on top of SERP when it’s really relevant. Or we also show like lower down the page where you might have different intent. ⁓ But still it could be useful if it’s something that you scroll down to.Gabriel Yeah, absolutely. ⁓ And I want to get into the, you mentioned it being concise and that’s one of the main differences I feel we have with Google and I want to get into a bunch of those. ⁓ before we do that, let’s continue with the basics. I so I actually started working on this feature. Maybe people think it was in reaction to Google or something, but it wasn’t. We actually started working on this as soon as Chat TV came out. ⁓ to really initially focus on Wikipedia and helping, know, giving people better Wikipedia answers, which you mentioned, search exists now appears in about 25 % of searches, Wikipedia appears around about 10%. So, and we had gotten lots of feedback over the years that, hey, it’d be great, you’re showing me the beginning of Wikipedia, but it’d be great if you could just show me the answer from Wikipedia. And we had tried that in different ways and we accomplished that somewhat, but until LMS came along where we could really pull back the paragraphs of Wikipedia and ask the LMP to pull out the answer within that paragraph, which is much better than just showing the paragraph and making you find it. We weren’t really able to unlock search assist. So yeah, we started working on it right when ChatGPD came out and kind of rushed to Wikipedia. And then I know you got involved later. when we started adding lots of other sources. And as you mentioned, we’re trying to use the best sources we can. ⁓ But I’d say more broadly, given that it’s kind of a broader thing than Wikipedia now, what do you see the problem that it’s solving in search results, just kind of for at large? Like you mentioned, sometimes you put it on top, sometimes you put it on the bottom. Obviously, that’s a choice. we’re putting on top because we think it’s solving a search problem,Ewa So think the key problem is that it’s short that we’re solving with assist is that it shortens the path from when you know what you want to ask and you formulate a query and to actually finding what you’re looking for. And to your point, we’ve already been doing Wikipedia or other modules in the past and we’re still doing them for many of the searches. But Assist allows us to cover more of these informational queries, including the long tail ones, meaning people use different language to ask Search Engine about what they’re looking for. And with Assist, we’ve been able to understand more of these natural language queries or queries that really ever are asked only once to a Search Engine, which is a huge portion of search queries. And, but we still can understand them and can present an answer that is like good enough to answer what you’re really looking for. But also with the more button that you’ve already mentioned also allows you to dig deeper and get more information on demand while still keeping you in the search engine context. In the context that a lot of people are familiar with because we’ve been using it for years, years, some of us ever since they were born. And so it’s kind of like bridges the gap between the value that LLMs bring and how they can enrich the experience of finding information ⁓ without having to move to a totally different user interface, to move to more like conversational chat experiences. It’s still search results that are familiar. It’s the search results that these answers are grounded for. but we’re making use of this technology to present it in a more suitable way for larger volume of different types people ask search engine.Gabriel Yeah, and so maybe I summarize that way. It really is saving people time. And I think as a primary benefit, I think as a secondary benefit in aligning with our vision of Raise a Standout Trust Online is that we’re trying to ⁓ understand what is the best information in the search results and surface that for you in a concise way. ⁓ So that not only saves you time, but it on average should be giving you better information higher up on the page, ⁓ which is kind of really what you want in a search engine. And just to restate for people who really don’t want AI, you can turn it off. However, this is not to your point earlier, AI making up the answer. This is us grounding the answer on actual search pages that were crawling in real time to look for that answer for you. And then the sources are ⁓ annotated there, which you can click through ⁓ and both check and read more information because we’re only giving you a concise summary. So if you want more, you click through. ⁓ So with that in mind, how you mentioned the more button in the UX before we kind of dive into kind of differences. ⁓ What is the general user reaction been like over time with search assist?Ewa ⁓ So we’ve been getting a lot of really positive reactions from our users. Assist has been like one of the highest rated parts of search experience historically at DuckDuckGo. And ⁓ I think what people usually appreciate is both that it saves time, it gives this concise answer. The fact that it’s really concise and it’s not like taking over your search experience is just there when you need it but still doesn’t make it hard to get to organics if that’s what you’re looking for. It’s another thing that people have appreciated. ability to drill down to sources, as you’ve mentioned, is also something we’ve heard been ⁓ getting like really good reception. And we are using the feedback we’re getting from people a lot in improving assist. On one hand, that’s because we really don’t track our users. We have very little information about how people interact with our search results. So we really rely on when people make effort to click thumbs up, click thumbs down, leave some additional comment. We use this information both in automated way to improve our answers and also we really do read through them and take lessons and figure out how we can continue improving assist, which is for instance how we’ve gotten the more button.Gabriel Yeah, I mean, so that seems like a big difference from Google right there for what it’s worth. I mean, I guess I’m not inside Google, so I don’t know, but from reading comments on Hacker News and other places, it does not necessarily seem like they read every piece of feedback, ⁓ but we actually do. And so ⁓ that really is a distinction. ⁓ We mentioned some others too. ⁓ So I mentioned that it was optional. I think we should clarify now that it’s not just you can turn it on or off, which you can. But you can also change the frequency of when search assist appears. We have often and sometimes, as well as on demand. So you can basically make it so it doesn’t show up automatically, but if you want to click on it, you can still click the search assist button on the underneath the search box and it’ll show just for when you want it on demand. And if you really, really don’t want it, including not even seeing that button on the page, you can ⁓ get rid of that. So that’s another one.Ewa Yes.Gabriel Another one that you mentioned, so the conciseness, I think, you know, does a couple things. One, it means that the less information there is, the less kind of surface area there is for making stuff up or getting stuff wrong. ⁓ But also, what you had just mentioned, it just takes up less space on the page. So I think some complaints that people have about Google’s A.O. overviews is they really are just taking up. You can’t see the organic links, they’re taking up too much of the page. We’re really keeping it ⁓ tight. And the way we’re doing that is we’re kind of forcing it to be small, but also to your point, we have the more button there as an option, but we’re not starting out with that long explanation. We’re starting out with this ⁓ concise explanation. Is there anything else you want to highlight in terms of kind of differences between us and Google? ⁓ If not, I can probably come up with something else.Ewa Maybe like building on top of what you just talked about is I ⁓ think what what I can tell about our philosophy of building assist which I think also applies to the whole of search is that We like to give control to our to our users. We trust that they know they can own their experience, which is why we’re giving you search results, which some of it is assist. You get the organics, you get other modules, none of these experiences is like overly dominating the search results page. And you can kind of like choose your own adventure, which I think is... Again, I’m not in Google. I don’t know exactly how decisions are made in there, but from my observation is like different approach to how we give control versus deciding for the user what they should be seeing when they search for a specific topic query.Gabriel Yeah, that’s a great point. I mean, I would even build on that, which is we’ve also been giving control to publishers. There are some publishers who don’t ⁓ want to be part of AI results. ⁓ We are trying our best to make the click through rate actually pretty decent because we have a short summary. And so really, if you want more, you click through and the links are prominent. Like on Google, sometimes they’re hidden actually behind the click and on DuckDuckGo they’re not. More fundamentally in that if you do want to opt out, we have a kind of way for publishers to just opt out of the AI summaries ⁓ via robots.txt ⁓ and still appear totally fine in search results. Whereas Google has been trying to bundle those things together and basically force you to opt out of Google altogether. ⁓ So that’s another decently big difference in line with control. ⁓ OK, so yeah, what’s What’s next? What are the negative feedback complaints we’re getting? And what are we working on? Where are we going?Ewa Yes.Ewa So ⁓ one of the things that we continue working on and we will... is that we want to keep the high bar for answers quality. There are still cases where we get the answer wrong, it’s not incorrect or maybe the sources weren’t right or we misinterpret the query. The percent of these incorrect answers is really low but we, as I said, we want to really take correctness seriously. So we’re working to add even more loops, both with user feedback, observing the queries, especially the fresh ones, like trending topics. This is where information changes really often and we want assist to always give you the fresh and correct answer. So definitely answer quality is something that we will continue working on. ⁓ We also are moving to other languages besides English, as it has been available in English for a while already. That hasn’t been true for other languages, so it’s already available in Italy for some queries. We’re only rolling it down. You’ll start seeing it rather further down the page before we build enough confidence to show it on top of your results, but you can already start seeing it there. And we plan to roll out to ⁓ a few next languages which are gonna be Spanish, Dutch, French, German and we won’t stop there but this is like the current focus which ⁓ I’m personally excited about because yeah it’s...Gabriel sure a lot of team members are. have, I mean, this is for InterObject.co, we have people, I think, across like 30-something countries now, and a lot of them are not English as their first language. And I think they would prefer to have non-English ⁓ search assist answers. So that’s exciting.Ewa Absolutely, but also our users. I think we know that people all over the world are using DuckDuckGo search and I personally can’t wait to have them use assist and also get benefits of these short and concise answers. And we’re also looking into improving and expanding or continue improving the user experience of assist. We’ve been already mentioning the more button a few times. We know that there are sometimes different expectations people have when clicking more buttons. Sometimes it’s really you want to have the longer version of the answer, but sometimes it’s more like you want to do more information or additional thing that wasn’t mentioned, the concise answer. Or maybe you want to drill down, more like learn about related things. So we’re now looking into exploring these options and making sure that expand answers are really helpful depending on what you’re looking for.Gabriel Cool, I know we’re also always trying to speed things up. ⁓ reducing latency, getting answered faster, it’s always on the list. ⁓ Yeah, I guess the only thing else comes to mind is, you know, we’ve been, it might be worth pointing out is, which we mentioned in the past, that this actual ⁓ technology stack is completely independent from being in Microsoft. ⁓ You know, we’ve been kind of, We’re obviously working with the LM partner. We’re not building a foundational model ourselves, but the stack, ⁓ you know, is coming via our own crawling, ⁓ the web pages where we’re generating the answers. And that index is improving every day. And so we’re working on that as well. ⁓ And ultimately we hope to say more about that. ⁓ I think that’s it. I mean, the, ⁓ Anything else you want to add or do we hit everything for the basics?Ewa we hit everything that maybe one thing which is interesting to me, I don’t know if it’s going to be interesting to others, but ⁓ I think it’s also worth mentioning how we determine, for instance, answer correctness and how we’ve been actually using a lot of manual reviews of queries and answers within the team, which ⁓ I think is not obvious. Some people might assume that if you’re using AI generated also you also delegate to the AI to determine if it’s correct or not or if it includes enough details or not too much details. So the way we’ve been building assist is that we’ve done a lot of that manual judgment ourselves and we rely on these manually reviewed datasets. I’m pretty sure quite a few people in the team ⁓ might complain a lot about how tiresome and ⁓ maybe you it’s It’s not the most exciting work while you do it, but I think it’s extremely useful in really ensuring that these answers, when we say our correctness rate is high, this doesn’t mean that someone or AI tells us we’re doing a good job. It’s really, we’ve reviewed a huge, huge, huge data sets of queries and we retest them over time to ensure that this quality stays high.Gabriel Yeah ⁓Gabriel That’s good, that’s a way to end. I guess I have been forgetting to say, if you’re listening to this and you do have follow-up questions, email us. You can reply to the email that you get if you subscribe to us on Substack. Otherwise, we probably need to develop an email address that other people can send to us. So I’ll have to get back on that. But if you subscribe to the Substack, you can just reply to that email and it’ll come to us and we’ll get it for future episodes. So thank you everyone for coming on and thanks everyone for tuning in.Ewa Thanks, Gabriel. Thanks, everyone. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit insideduckduckgo.substack.com

In this episode, Beah (SVP, Product) and Zbig (Director, Talent) discuss our approach to hiring, and how it’s designed to reflect our unique, cross-functional and mostly async ways of working. Show notes: Check out our careers page and open positions here. Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.Beah: Hello. Hi, everyone. Welcome to DuckTales, ⁓ where we go behind the scenes at DuckDuckGo and discuss the stories, technology, and people that help build privacy tools for everyone. In each episode, you’ll hear from employees about our vision, our product updates, our approach to AI, or how we operate as a company, which is the zone that today’s episode will fall in. ⁓Zbig: Okay.Beah: introduce myself briefly. I’m Beah Burger-Lenahan. I lead the product team here at DuckDuckGo. And we have with me Zbig. Hey Zbig, do want to introduce yourself?Zbig: Yeah, of course. Hi, Beah. Hi, everybody. I’m Zbig. I’m on the Talent Acquisition team. I’ve been around for almost seven years now at DuckDuckGo, and I feel very privileged to have seen this amazing journey of us scaling the team from about 50 to over 350, where we are right now and I’m super excited to be here and talk about hiring.Beah: Awesome, thanks, the big. Yeah, so today we’re gonna talk about how we hire, what that process looks like, why, and so forth. Hopefully it’ll be interesting to everyone. ⁓ So maybe just to get started, can you kind of lay out at like the highest levels of our approach to hiring, what that process looks like?Zbig: course. Yeah. So I think at a, at a highest level, um, there is, you know, lot of intentionality and discipline to how we hire. So we would do that only when we have like a really clear and well articulated need. So when we’re scoping roles, right? Like we would want to deeply understand what the actual tangible problems are that we want that particular role to solve for us. And then we designed the hiring process to test for the skills, the competencies that are critical to being able to do that. So in practice, that means that we base that hiring process ⁓ largely on test projects ⁓ rather than interviews. Well, there are usually a couple of interviews in every process, but we definitely attach way more weight to test projects. We also tried to design that hiring process in way that ⁓ it’s truly like a two-way street and allows Canada to discover how we work. So, you know, they can make a well-informed decision at the end about whether or not this is good place for them. And I think one other thing worth highlighting ⁓ that comes out of this intentionality and discipline is the fact that, you know, never in the company history, we were forced to do any group layoffs. And when you look at the tech market these days, that’s very rare.Beah: Yeah, because we only hire people that we know we really need.Zbig: that will really advance the goals of the company, right?Beah: Yeah, got it. Okay, thanks. So let’s talk about the projects since they are, as you said, kind of the core of our process. Are these projects, do we expect people to do them for free?Zbig: Sure. Yeah. No, we offer payment, which is kind of like average bubbly, but it’s usually like anything from 50 to 100 US dollars per an estimated hour of work required to complete a project. These are, and this is just like recognize the time candidates put into those. These projects are always role specific and designed to simulate the actual work one would be doing on the job. You can complete them async on your own time. And I truly believe that allows candidates to demonstrate the best of their abilities without the typical pressure and stress of interviews. And also like when you think of the async format, that much more closely reflects how we work on a daily basis at DuckDuck. Because I think that probably like, I don’t know, 70, 80 % of collaboration here happens asynchronously. There are a few meetings throughout the week. And ⁓ we’ve picked this format for the hiring process very intentionally. There’s actual research behind that that proves that work samples, test projects that are role specific, they’re much less prone to bias and error than interviews. And what we found through that ⁓ is that they are great predictors of future success on the job. And when you look at our retention rate, that’s 95%. You know, our engagement scores are also very high, like I think 86 % way above market benchmarks for companies of our size and at our stage of growth. like there’s actual, you know, quantitative validation that this approach works very well for us.Beah: Yeah, is it challenging to get people to commit to doing these projects? It can be a bit of time.Zbig: It is one of the bigger challenges of our hiring process because these test projects can take anything from like three, four hours up to 15 and in some cases maybe 20. And we typically expect folks to complete two of those, two sometimes three. So that’s like a time investment of about 25, 30 hours in test projects. I think we do our best to really to adjust to Canada’s availability. There’s like from our side, there’s no pressure on like when exactly they should be starting on those. They can plan ahead, set that time aside. If they need an extension, they can just let us know. And we usually have no problem to grant that because we understand that different things can come up in life that could derail their plans. So So we really try to adjust our pacing to candidates, be very flexible with those timeframes to ⁓ make it feasible for them.Beah: Got it. I’m shifting gears a little bit. So where do we hire in the world and why?Zbig: Hmm. So we hire globally though, not everywhere. We have like a selection of countries where that we’ve decided scale to based on like ⁓ a pretty nuanced estimate on the available town pools, how those relate to our needs, ⁓ how easy it is also to like scale within them from the administrative financial logistics perspective, perspectives, but we are able to hire across tens of countries around the the globe. And I think that’s awesome because that gives us access to amazing talent ⁓ that often when you think of some of these countries, there aren’t that many opportunities to work on something of a comparable scale that Go offers. So we’re way more competitive there, right? And we’re truly... I’m actually in Krakow in Poland and we have a pretty pretty big contingent here, like I think one of the fourth, fifth biggest representations geographically across the company with over 20 people. yeah, there’s definitely a great advantage of this remote setup that enables that. ⁓ And I think also what makes it possible is the fact that as a company, we’re designed from ground up for full remote collaboration, right? When you think of the companies that were switching to remote during the pandemic, that often didn’t work out and many of them are now calling people to go back to the office because they didn’t really have the right processes and culture to enable effective collaboration. I think that’s definitely not the case here.Beah: Where are you located today?Zbig:Beah: Yeah. Yeah. I will say, mean, the remote ⁓ employee base has challenges for sure, and it has a lot of upsides as well. I think for me, like one kind of just fun perk of it is that it’s cool to know people from all over the world. ⁓Zbig: yeah, for sure.Beah: you know, when I think about like traveling I think, ooh, like who can I go see?Zbig: Yeah. And also when you think of like, we’re building a global product or a set of global products, right? And I think it’s invaluable to have people from different geographies representing different cultures and perspective, because that helps us inform like how we can be building those products for a broader audience that really ⁓ meets their needs and solves jobs for them.Beah: Yeah, makes sense. So you mentioned ⁓ one of the challenges of our hiring process is getting people to commit the time to the projects. Are there any other significant challenges?Zbig: Mm. Yeah. I think the probably the biggest one is related to how we work. Cause as you know, we’re pretty uniquely organized over here. There’s no middle management. There’s no like separate project management function. And in practice, that means that everybody really is expected to be able to scope, propose and execute projects. Sometimes that means, you know, managing a cross-functional project team and that skillset is not that easy to get on top of the functional expertise, Like functional competencies. So ⁓ we often end up hiring, you know, we do most of our hiring in engineering and at a senior individual contributor level, like senior engineer, and we often end up hiring folks, you know, performing these like more senior leadership related responsibilities elsewhere, sometimes even like holding more senior titles than what, we have on the job description. Disclaimer though, don’t use job titles internally. We don’t want them to get out of picking the best solution or going in the most optimal route. We try not to make decisions based on authority. And that’s worked well for us, I think, over the years.Beah: . . I’m curious. I’ve never asked you this question or I don’t know the answer. Do candidates find it appealing that we don’t use titles internally or does it put some people off?Zbig: Thank I think many of them, I think many of them do, and they do highlight, well, we get a lot of that feedback from candidates in the hiring process, but some of them do raise it as an objection. And I get it. you know, I think typically on the market, there’s a lot of weight attached to job titles, right? Because they demonstrate certain progression throughout one’s career. And some candidates decide not to give that up, which I totally get, right? Like if someone is already like, I don’t know, staff. engineer, principal engineer, and the max they can get here with us is a senior level position. ⁓ I understand why, you know, a certain percentage of people would not want to do that, right? But in practice, though, like for people who really care primarily about the ability to make impact in the org, that title here, it doesn’t... doesn’t matter really, right? Like it does not give you more authority or impact in any way.Beah: Yeah. So I think like in our conversation so far, a lot of it has been about this like very particular point of view that DuckDuckGo has that ⁓ may be different from a lot of companies, a lot of hiring processes.Zbig: Mm-hmm.Beah: ⁓ But my sense is also that we have evolved, like while we have a very particular point of view, it is not a stringent one that has been the same for years and years and years, like it’s evolved quite a bit. Any significant changes in your time here that you think are worth calling out?Zbig: yeah, yeah, of course. And I think at a higher level even like we continue to iterate and improve all of our processes, right? ⁓ And that applies to hiring as well. So like one of the big things we had in the past was the so-called internal contracting period when, you know, once you completed the hiring process, we didn’t bring you in full time, but we asked you to do some sort of a part-time internal contract. you know, do some you know, a single sometimes maybe two projects as part of the team, but not working with us full time. So usually people, I did that too. It wasn’t the best experience. So the first thing I did after joining was to look at the data of like, okay, how many people going through that actually join, right? Like how big of a filter is that in our hiring process? And what it turned out was that, well, we actually bring in everybody. Like there was, think one exception that that we weren’t even like sure about going into that internal contracting period. But the data told the story that, well, the hiring process itself is such a good predictor already. We don’t need that. And that was a, I think that was a great change because it enabled us to engage with people who otherwise would not be able to invest that time on top of their jobs or personal obligations, right? So that was definitely a big enabler in accelerating the pace of hiring.Beah: I did that. ⁓ back. Got it. So we did that, that was like inspired by you not having a good experience, like the con, you being kind of put off by the.Zbig: Not just me, There was a broader, like my experience was one of the data points, but there was a broader discussion with leadership where, you know, we looked at like, okay, how scalable is that? Like we’re hiring maybe 20, 30 people per year. We were when I was joining, but we were looking at increasing that substantially, right? So it wasn’t a great experience for candidates. wasn’t a great experience for people internally. Ask to oversee those internal. contracting period, so there were multiple arguments for making that change. But only when we examined the data, we knew that, OK, that’s up. It doesn’t make sense to be doing that anymore.Beah: Mm. Yeah, that makes sense. You mentioned we were hiring maybe 20, 30 people a year at that point. Can you talk about what our hiring volume looks like now?Zbig: Yeah, so it’s our record year this year. So I’m super pumped about that. We are on 80 hires this year, ⁓ which is absolutely amazing. Yeah, yeah, there may be a couple more. So we’ll see. ⁓ But yeah, it’s really great to see that growth. And we’re bringing in some amazing people. So yeah, that’s super cool.Beah: And we got 2 months left to go. Yeah, do you think we, over the process of the last few years, our ability to make the right selection in terms of both people that will be successful here and that will be happy and thrive and want to stick around has improved?Zbig: I think so, yeah, and quite considerably. ⁓ So we were looking to tighten the feedback loop for new hires to get an even better sense of how fast new hires are reaching full productivity. But when we look at the data, ⁓ the success rate is very high. And ⁓ we rarely have instances of new hires failing to meet expectations, which is another benefit of this highly disciplined, intentional approach. ⁓Beah: Yeah, you mentioned 95 % retention. Yeah, that seems very good.Zbig: retention. Yeah, that’s continued over the years, right? Like when we scaled from 50 to over 350, we’ve managed to maintain that. And I think we’re only getting better.Beah: Yeah, I remember when I was in a much loosey-gooseier process to be hired at DuckDuckGo seven-ish years ago. was talking to Gabriel and I was kind of asking him about what it’s like to work at DuckDuckGo and he was like, mean, nobody’s really left in the last four years. It wasn’t exactly what he said, but was something like that. Basically nobody had left in years, maybe like one or two people. ⁓Zbig: Right. Later. Yeah, yeah,Beah: Okay, one more question for you and then ask if you have anything else you want to say before we close. what, I’m curious what proportion, where are coming from? ⁓ What portion of hires are referrals versus inbound?Zbig: So the vast majority comes through inbound, and that’s a combination of our Curious website and ads we post on LinkedIn and other job boards. But Curious is still, I think, the main contributor there. It’s, I think, 64 % for inbound total. And that makes sense, right? We have such a big user base where pretty popular among engineers. A lot of our hires are actual users of our products, which is amazing because then they contribute with their feedback and thoughts to helping us improve our products. ⁓ Similar to other companies, ⁓ the second best source of hires is referrals. I think a quarter of our hires ⁓ come from referrals and ⁓ that’s always been a great source also in terms of the retention and performance of these folks. So we’re trying ⁓ to leverage that as much as possible.Beah: Also, just interject that when we hire people who are not users of our product. They’re also really great at helping make the product better because my question is, why aren’t you a user? What’s stopping you? I mean, we expect everybody to use our products once they’re here, but it’s sort of nice to have people who maybe like weren’t dedicated, loyal users ahead of time because they maybe have a different basis of comparison and also reflect a large number of users that we would like to ⁓ list.Zbig: For sure. Of course. Yeah. I think. Exactly. I think it’s great to have both of these perspectives, right? ⁓ Because the non-users, they usually bring a completely new set of eyes and views on our product offering. So yeah, I think we’re really contributing from having people from both of these poles.Beah: Yeah, what anything else that you want to share before we wrap, Sabing?Zbig: I think, yeah, I mean, like one other interesting thing is that it does take a lot of effort to hire, you know, and we have multiple people from our functional teams, engineering, other teams too, involved in the hiring process to help with test project reviews, to help with interviews. And perhaps for this, I looked up the data and, you know, only this year we’ve done over 2300 project reviews and 790 almost 800 interviews. So yeah, it’s a huge effort that goes into that. But we honestly think it’s worth it. Like getting hiring right and making sure that people who bring in will make a positive impact that will be happy here. I strongly believe this is one of the most powerful levers we can be pulling as a company. I think that effort is totally worth it. And of course, we continue to look at how we can optimize and improve the efficiency of the hiring process so it’s less of an effort. But that’s a continuous process,Beah: . Okay. Yeah. Awesome. Well, thank you, Zabig. That’s all I got.Zbig: Awesome. Well, super happy to be here and thank you for this. That’s been great.Beah: Alright, see you later!Zbig: See ya. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit insideduckduckgo.substack.com

In this episode, Gabriel (Founder) and Chris (Subscription team) discuss why we built the DuckDuckGo Subscription, its four features, and how it protects more of what you do online. Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.Show notes: Learn more about the DuckDuckGo Subscription here. Gabriel: Hello everybody. Welcome back to DuckTales, everything DuckTale go. I’m Gabriel, founder, CEO. I got with me Chris. Do want to introduce yourself?Chris: Yeah, hi, I’m Chris Calvi, ⁓ long time listener, first time caller. Had to say that. ⁓ But ⁓ I’ve been at DuckDuckGo for a little over four years. I’m on the partnerships team here. And ⁓ I guess I’ll take a second to talk about the partnerships team, what we are. ⁓ We are primarily the team that handles all of the relationships with third party companies that DuckDuckGo works with.Chris: We’re a relatively small team, about 10 people. you know, classic examples of companies that we’d be working with are any of the live information that you’re seeing when you search DuckDuckGo. So that would be like sports scores or stock quotes, weather, that sort of information, also like flights and lyrics, all of those relationships, we get that information, we license it from...from other companies. that would be an example of work we would be doing. And then also, we will even work on things like infrastructure partnerships with cloud partners and AI companies and all that stuff. So that’s what we do over here at Partnerships more recently.Gabriel: Interestingly though, the way we work, and this gets more inside deck to go since the name of our blog on this, we don’t really work functionally though. We have objectives inside the company to get a particular thing done, and that thing usually involves many different functional teams. What that also means is those objectives have owners and they can be from any functional team. ⁓You tell me how you think of that, but I think if you do what doing recently actually was not very partnershipy. You’ve been helping launch different parts of our Dr. Go subscription, which involves some partnerships, but like your day-to-day job is not always partnership related.Chris: Yeah, mean, absolutely. And I think you as people, if they listen to a bunch of these episodes, they’ll see that. People might be leading an objective and they’re on the design team or in this case, the partnership team. A lot of times it is somebody from the product team. And in this case, I’m working a little closer on the product side. So you’re right.Gabriel: And yeah, so we’re going to talk today a little bit more deeply because you’ve been working on it about the DuckDuckGo subscription. We’ve mentioned it on some of these episodes, but haven’t really given a big overview. So that’s what this is going to be. The subscription is a bundle. So it’s got a bunch of different things in it, which maybe you can get an overview in a second. But just to say that we might have separate episodes about going deep into some⁓ One of those things are a technical aspect to them, but this is we can give more of an overview of kind of where it came from and what it exists today and kind of where we’re headed with it. You want to start and just tell us kind of like what’s in it and how it works.Chris: Yeah. Yeah, that sounds good. The DuckDuckGo subscription is a relatively new thing for us. We launched this in April of 2024, so it’s really only been around for a year and a half compared to the search product, for instance, that’s been around for a very long time. And what it does is really, I mean, in a nutshell, it supercharges some of the existing functionality you have. It gives you extra privacy features within your DuckDuckGo ⁓ app and experience. ⁓ It does a little beyond that too, which we’ll talk about, but that’s the key thing. ⁓ The other thing I will note is that if for long time subscribers, you may have originally remembered it being called Privacy Pro. That was the original name for it. We recently just changed it to DuckDuckGo subscription for to keep things simple. So we’ll call it the subscription for the rest of the episode. ⁓ But before I like...Gabriel: Yeah, give us the basics. yeah, what’s in it? Yeah.Chris: Yeah, the basic things, the four basic things in there are the, you got the VPN, which we’ll talk about what that is in a second. You get access to advanced AI models within Duck AI. You ⁓ get personal information removal, which I’ll talk about what that is, as well as identity theft restoration. But before I go down all that, I did want to ask back to you about ⁓ why we came up with a subscription. and then I’ll kind of unpack each of those core pieces.Gabriel: Yeah. Yeah, I mean, the way I think about it more broadly is we want to be your clean internet experience. Like if you want a more protected way to operate online where you’re not getting followed by ads and you’re not giving up your personal information, you kind of adopt the DuckDuckGo ecosystem. And so that’s our browser, it’s our search engine, it’s our AI product. And generally we want to offer those essentially for free. mean, there’s advertising model in the search engine that pays for this, but we can, we want to give it for free in the sense that mo because more people can use it around the world. However, there are certain protections that costs us a lot of money to do that we hadn’t been able to offer because of that. So VPN is the obvious, a canonical example there, because when you turn on the VPN, now all your bandwidth is running through that VPN and a lot of that’s video. ⁓ And it’s just not something that can be advertising supported. doesn’t, the numbers don’t add up. And so we’ve wanted to offer that because it is a key part of being fully protected for people who want that kind of fuller protection, but we couldn’t offer it for free. And so we needed a subscription. At that point, I thought, hey, we could just offer a VPN, but...I think there are other things in this category that have real marginal cost in business terms. ⁓ And it would be great if we could bundle them all together at one low price and allow DuckDuckGo users who really want more protection ⁓ to be able to get it in one place. You don’t have to sign up for multiple things. And that’s where the subscription came from. ⁓ I think we’re living up to that. hope over time we add more and more things and make it more and more valuable without increasing the price or much at all. ⁓ But yeah, that’s where it came from.Chris: Yeah, I mean, that makes sense to me. to dive into the four pieces of functionality, I’ll start with VPN, because you mentioned that. We probably should give a little bit of a high level of what a VPN is. So it stands for virtual private network. But what that means is that we basically have this anonymous private secure no logs VPN server that’s sitting out there, right? And we have these in 40 locations around the world. You likely are connecting to the one closest to you, but you can change it to go to another country. But what it does is all of your internet traffic, the requests that you’re making to, for instance, as you browse the web, are gonna be securely essentially tunneled through that server and then sent out to the resource where you’re requesting that information. So the website that you’re accessing,they’re not going to see, under normal conditions, without a VPN, they would see your IP address. In this case, they’re going to see the DuckDuckGo server’s IP address, which is meaningless, essentially, to them from a privacy perspective. So it preserves your privacy on that front. then they’re sending that information back, and then it’s coming back to you. So then the second place where it’s adding privacy is on your own, sort of like your internet connection. So if you’re...⁓ you’re making that request in a lot of cases your internet provider or if you’re using like a public Wi-Fi, they can see the sites that you are accessing and sometimes can see the information that’s coming back and forth. And in this case where it’s that since it’s all being encrypted and coming through the duck, going to DuckDuckGo server first, they’re only seeing that you’re connecting to this one server and that they’re not seeing any of the data in between. So it gives you sort of that two different protections when you use a VPN. I don’t know if I’ve done it justice.Chris: Yeah, I think you did a great job there. The only thing I would add to that is, yeah, so it really shields your IP address and other information from leaking to both your internet provider and the sites that you’re visiting, as well as all the places in between too. People may not realize like when you route across the internet, you connect to something and then you connect to like a bunch of different hops in between lots of other internet providers. And it shields not only from the end points, but everything in between.⁓ Additionally, the IP address is often used to get your location. ⁓ And so by effectively shielding your IP address, you’re effectively shielding your location. And the final thing I would add is like, it may seem esoteric, like do I need to shield this stuff from these people? ⁓ But it’s been well documented that ISPs, internet providers are selling this data all the time. ⁓ And so...If you want to be maxing protected, you do want to have the VPN. ⁓ And you know, I leave it on all the time, our VPN. The other thing, the final thing I would add, which you know, doesn’t relate to the privacy necessarily, but it kind of relates to it in the sense that trusting us is that we decided not to sub license this VPN or anything from anybody else. We are operate, we made this from scratch. We’re operating all the servers ourselves. ⁓ And so it’s fully DuckDuckGo operated.Chris: Yep. ⁓ Thank you. ⁓ And then on the kind of jumped into the next feature that I had mentioned was the advanced ⁓ AI model. So you get in Duck AI. So ⁓ Duck AI is free. You don’t need an account to access it. And it’s our AI chat ⁓ product. But if you’re a DuckDuckGo subscriber, you get ⁓ access to advanced models and higher limits on the models that are available for free.So the advanced models sort of the difference between the advanced models and the free models Well the advanced models just to name them are ⁓ open AI’s GPT 4o GPT 5 Anthropics ⁓ Claude sonnet 4 and Metas llama for Maverick and So we have these are available if you’re a subscriber and then we have the free models which are available to everyoneThose free models are great for lighter weight tasks that are optimized for speed, whereas these more advanced models are made for more complicated prompts and more challenging scenarios. I don’t know if you have anything to add on to that.Gabriel: No, that’s a good explanation. The only thing I would add would be it’s for the same reason like the VPN, like these advanced models cost a lot of money to run. when you’re the computation on advanced models, it’s more computation, more computation, it’s more cost. And so we just couldn’t offer them for free. But as part of the subscription, it’s a natural fit. if you’re paying for it, and we haven’t said the price yet, so why don’t you explain what our pricing is?Chris: Yeah. So.Gabriel: But if you’re paying for it, we can afford to give you this. But we’re still trying to strike a balance here and give. We’re not trying to make tons of money off this. We’re trying to give a good price for the bundle. And so we think we’ve been pricing it competitively.Chris: Yeah, absolutely. I should, yeah, let’s step back. the thing I’ll say is it’s available in 30 countries today. So the U S Canada, UK, and all 27 member countries of the European union. And you, the price in the U S is nine 99 a month and it’s nine or 99 99 a year. So basically $10 a month or a hundred dollars a year if you sign up annually.⁓ And then it’s that’s basically in your local currency if you’re subscribing from one of those other countries ⁓ But you can get a free seven-day trial too. So ⁓ That’s that’s important to note so you can give it a spin See if you like it and then if you do like it you can continue onGabriel: Got it, okay, so we have VPN and DuckAI advanced models. Now as I understand that those are available in all those countries, the other two features are more US, or there’s one of them US specific and one of them we need to talk about how it might differ, right?Chris: Yeah, yeah, let’s talk about personal information removal first because I think ⁓ that one is US only. And the reason for that is, you could probably go on about this, it’s primarily a US only problem. And so the way that, what happens if you’ve ever searched, for people listening, have you ever searched the internet for their name, which I’m sure everyone’s done at some point you’re going to see yourself show up in, especially if you type in your name in your city, if you have a less common name, you’re going to see a bunch of stuff show up, lot of of spammy sites that have your address, maybe your phone number, email, some other information, maybe socials. This stuff, ⁓ they are selling essentially, and in the US, a lot of countries make that illegal, in the US it’s not. So ⁓ what we will do is with personal information removal is,We help you go out and on your behalf, we opt you out from I think at least 80 sites like that today and we add more all the time. And so that’s how that works. Yeah, but I don’t know what else you have to add onto that.Gabriel: No, you’re right. I mean, that’s why it’s in the US because it’s just not a problem. It’s not really a problem outside of the US. I mean, the only thing I would add would be this is one of the reasons why people are getting tons of spam, texts and junk mail previously, now lots of, you know, everyone’s got a phone, spam, texts. And so if you remove yourself from these sites, that will reduce as well as potential for identity theft. And spam emails too, sometimes the emails out there. ⁓ In any case, ⁓ it’s US only, but if you’re in the US, it is a real problem. And I suggest you do something about it, if not from us, from somewhere, because it really does make a difference if you remove the things. Now, the other thing I would say, and we’ll probably do another episode of this, is that these sites are often, like you said,they’re buying and selling information from other sites and other places. And so if you just remove yourself once, your information will come back on these sites eventually because they ⁓ just buy it again and get it from somewhere else or from public record or something like that. That’s another problem with the US is they get it off from public records. like you like buy a house or something and they end up buying that information. ⁓ But that’s why you want a service to do it because we are constantly, the service is monitoring it and scanning these sites repeatedly. And so if your information shows up again, three months later, it’ll get noticed and then we’ll submit another removal request to remove it. So it’s kind of like always working in the background to remove yourself in an automated fashion from all these sites that we cover.Chris: Is it worth talking about kind of how that information stays on your local machine in this?Gabriel: Yeah, I think so. I think so because it’s a differentiator and it took us a lot more time to build this because of that. But I do think it’s a privacy benefit and worth it relative to other sites that do similar. Yeah, you want to explain it? I can do it too if you want.Chris: Yeah, mean, well, when you said it’s happening in the background, what we mean is quite literally on your computer in the background. It’s not happening on your data, you know, because we need, in order to opt out, you need to provide your name, right? And some information so that the service can opt you out. So rather than that information living on our servers and then doing the opt outs, it lives on your computer and does the opt out.And to your point, you mentioned that’s a differentiator. And it’s like a core thing for us is that we don’t want your personal information. As much as possible, we just don’t want it. And so that’s, I think, was worth mentioning with this product.Gabriel: Yeah, I’m actually not familiar with any other product that does it that way, that where your information really doesn’t live on an account on the server. It’s on your, in this case, your browser. It’s built into the browser. I think there’s two reasons for that. One is to do that, you have to have a browser. So we have a browser. So we were kind of uniquely able to do this versus like a web service. And also it’s a really pain to build this, to actually get it to work in the background. I don’t think anyone would do this. unless they really wanted to get that privacy protection ⁓ maximum, which is what we generally do. ⁓ So yeah, it’s unique in that regard. ⁓ We should probably mention that that is desktop only for that reason, because it was very difficult to bring it to mobile. But we’ve been working on bringing it to mobile. And that’s kind of like next step for personal information removal is bringing it to our iOS and Android browser.Chris: Yeah, I mean, I’m glad you mentioned that because I know that that is a big piece of feedback we receive. A lot of our users use, they may only use us on mobile, they may only have a mobile device, and so they’re not able to currently use this functionality unless they get a desktop going and get it going on there. So it’s exciting news. ⁓If we pivot over to that last piece, the four, so there’s the VPN, advanced AI models, personal information removal, and then that last one is identity theft restoration. This is essentially in the instances where you are the victim of ⁓ identity theft. ⁓and you mentioned that it’s a little bit the function, that way that it works is slightly different whether you’re in the US or in another country. If you’re in the US, if you use identity theft restoration, they’ll be able to do ⁓ some of the, if you’re trying to restore your credit and everything like that, whatever was stolen, they’re gonna give you guidance on how you can, ⁓ you’ll connect with an advisor, essentially you’ll make a phone call.They’re going to help you, they’re going to handle your case, help you get your identity restored. In some cases in the US, they’ll actually be able to handle some of that work on your behalf. They may be able to contact financial institutions and things like that for you. ⁓ For identity theft restoration outside of the United States, they’re only going to be able to provide you the steps, essentially. The guide, like they’re going to say, this is what you have to do, then you’ll have to do it. So that’s one, that is a different it’s worth noting between the US version and the ⁓ version available to everyone else. ⁓Gabriel: Yes, and so like this, I mean the way I think about this is, you know, we’re really trying as a company with the product to cover all the bases for you, you know, online and protection in your life. And so we’re, you if you use our product, you’re really reducing your personal information out there, essentially to the maximum you can easily. But it can never go to zero, unfortunately. And if someone’s really trying to target you,Maybe they’re looking through your garbage, who knows? You ⁓ can be a victim of identity theft and there’s a lot of people who have been the victim of identity theft, unfortunately, it’s not a small problem, certainly in the US. I know more about the US, but ⁓ it’s a big problem in the US in part because we were talking about those laws earlier. ⁓ But if you are, it can really be a big pain. ⁓Gabriel: People can open bank accounts on your behalf and credit cards and spend money. They can mess up your credit report. And so what this is is essentially a peace of mind such that if that happens, we’ve partnered with this service that 100 % focuses on helping people restore their identity. So you will get a personal advisor to really walk you through this process so you’re not on your own. And they do it every day.you that’s their whole business is what they do is help people restore their identity. ⁓ And so by buying into subscription, you’re essentially buying the, ⁓ for this part of the subscription, the ability to have access to this advisor should you unfortunately find yourself in this situation, which we hope never happens. But if it did happen, you’d have this ability to help you.Chris: Great, yeah. That covers the kind of the four, I don’t know how we’re doing on time, but that covers our four pieces of function.Gabriel: We have no real time limit. This is longer than we’ve normally gone, but we have, this is a big topic, so we don’t have to go super much longer, but if there’s a few things or anything else you want to hit, we could do so.Christopher: I I guess, like, I’ll just kind of piggyback. You made that point earlier about the subscription is about providing the extra functionality that would normally, that costs money, essentially. The other stuff we’re able to do in an ad-supported way or in a free way, whereas this, you know, the things that we’re offering here, it doesn’t work. You need to pay for it, essentially. ⁓ what I will say is that it’s very complimentary.to everything that DuckDuckGo is doing on the free side. And we continue to grow and add new stuff on the free side ⁓ all the time. But what I’ll say is that if you are a current user of either our search or ⁓ DuckAI or our apps, this is, like I’ve found it to be ⁓ just incredibly complimentary. It sits very nicely on top of all of that. You turn it on.⁓ And to your point, you said you use the VPN all the time. I use it all day, every day. I’m using it right now on this call and I use it ⁓ on my phone all day. ⁓ And it’s really quite delightful to have that extra protection on top of everything else and to have the extra AI models, know, GPT-5. 5 Mini is in the free version, but 5, just 5 is not. And so you get that with the...with the subscription and so, yeah, think I’ll just end it there. Some of the stuff that I really like about the VPN is you can opt out, know, unlike for instance the Mac version, you can exclude certain sites because certain sites don’t play as well with a VPN, like ⁓ some streaming video for instance, like Netflix, you might run into issues where they’re not gonna stream it to you. So you can exclude a site or, ⁓in my Android app I can snooze, ⁓ which you can snooze the VPN for 30 minutes, which is really great for if you’re troubleshooting or if you can’t access your bank’s app or whatever through. So that happens from time to time.Gabriel: Yeah, these are good points. mean, yeah, we can wrap it up. I mean, I agree it fits nicely and is designed to fit nicely just on top of our regular stuff. And in particular, it’s built in, it’s mainly built into our browser, right? So if you get it, then you have some extra stuff going on inside your experience that you can get access to. I should also add that a lot of the things that you just mentioned, those are new features since we launched this description. So.We’re adding new free stuff all the time. We’re adding actually stuff to the subscription all the time. You know, we have, we’ve plugged a few times, but we have a quarterly update website, ducktico.com slash updates. I think it’s slash updates. Yeah, okay. And you can go through and look at like quarter by quarter, there’s a section for the subscription and you can see, you know, what we’ve been doing there, but like we’re constantly thinking about what to add to it. So it’s not going to be static. In fact,Chris: It is, yeah.Gabriel: the AI model aspect of it, you worked on, just got added recently, right? So we have more coming and ⁓ yeah, so thanks Chris, thanks for coming on. Thanks everybody for listening this long. ⁓ I think we’ll do deep dives. I think we should do deep dives on each of the four things in a future episode. I think those would make good episodes. Yeah, cause there’s so much to cover there. All right, let’s get out of here. Thanks everybody.Chris: Yeah, for sure. For sure. All right, thanks Gabriel. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit insideduckduckgo.substack.com

In this episode, Gabriel (Founder) and Esteban (Design) discuss AI chat organization, from automatic chat naming to ‘pinning’ your most used chats. Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.Gabriel: Hello, welcome to DuckTales. Inside DuckDuckGo features people, et cetera. You got me as the host again, this time I’m the founder and CEO of DuckDuckGo. And I got Esteban with me today. Esteban, wanna introduce yourself?Esteban: Sure. Hi everyone. I’m Esteban. Hi Gabriel. Esteban, I’m a designer in the team.Gabriel: Sweet. And that’s what we’re talking about today, Duck AI, and in particular, kind of new features we’ve been introducing around conversation organization that Estevan has been designing and working on. And we’ve been releasing kind of a few of these over the last couple of months. You want to just jump in, you want to share your screen and walk us through it. I realize we have some of this on audio now only.Esteban: Sure.Gabriel: Let’s also try to describe what we’re seeing while we do it. I’ll do the same thing.Esteban: Yes, for sure. Let me just share my screen and I’ll walk you through a few of the things that we recently dumped. Yes, so if you’re like me, you have tons of like conversations going on at the same time. What we have seen from users is that people who are very engaged with our platform, they end up having tons of conversations and then sometimes it’s harder to get back to them and to find the one that you needed. So we launched three things. The first one, I’ll show you an example. I was trying to see if I should ask you for a promotion during this podcast. Clearly no. ⁓ But the cool thing was that we, yeah, I will wait for a better time. ⁓ The first thing that we shipped was that we had ⁓ a title, the title of our,Gabriel: Good answer.Esteban: Chats were basically the same first prompt. It was just duplicated. That would make it harder to read. So the first thing that we launched was this automatic titling. We’re using the LLM to come up with a better title at the top. So it’s easier to find. ⁓Gabriel: So this is similar to how ChatGPT and some other competitors are doing it, right? So like we used to do just to repeat, so I understand fully, I think I do, ⁓ having reviewed this when it launched, but just to make sure. ⁓ Yeah, we used to just literally repeat the prompt as your title, like the beginning of the prompt. And so that it was kind of weird sometimes and duplicated sometimes, but now we run in the background something that... ⁓Esteban: Definitely Right.Yeah.Gabriel: Summarizes it for you and then automatically does this. You don’t have to do anything, right?Esteban: Yes. Exactly. And for example, I’m trying a new one. What day is today? That’s my prompt. And then immediately what I get as a title is day to day. The one thing is that sometimes the title is not exactly what you will call it. Like maybe day to day is not super descriptive. So we added this minor option, no? So we can say, I don’t know, day to day, it’s Thursday. Simple but useful.Gabriel: So if you want to do whatever you want, yeah, like if you don’t like our casing or something, you can put everything in lowercase or whatever, you can just rename it at will, yeah.Esteban: Yeah. Exactly. Yes, so after you have you want to name stuff, it’s very helpful when you have like several chats about something similar, but you want to have it personalized so it’s easier to find. So super simple, nothing that I don’t know, it’s blowing innovation in a way, but very useful. The second one is we noticed how ⁓ lots of our users were asking us for ways to save conversations and finding the conversations faster. We also have a limit of how many conversations you can have right now. There’s just 30 chats after the 30th chat. The next one gets deleted. We’re working on that, but we wanted to offer a way for you to say what’s important for you and why are the things that you want to come back to it. So I don’t know, this one. Let’s say it’s something that I was working on today, but I want to come back to it.add it to the top of the list by pinning it. So now it’s pinned at the top and it will always be there. That also means that if DocAI needs to delete one of your chats, it will go to the last one on the list and this one will not get deleted. You can pin up to five chats right now and yeah, it’s always accessible, always at the top, easy to reach.Gabriel: Got it, so it’s kind of the equivalent of like a favorite or a star or a pin in this case. All of these things are accessible from this three dot hover menu next to the ⁓ chat, chat to the individual chat. And then also you’re saying, is good, that ⁓ because the reason why we have the 30 limit at the moment is because all this is stored locally on your browser. It’s not actually stored on our servers. ⁓Esteban: Exactly.Gabriel: we’re working on an encrypted storage that we won’t have the keys to where you can get a much higher limits on it. But at the moment, or if you just want to keep it local, there is a local limit because your browser has storage limits. ⁓ But what this will allow you to do is keep ones around that you really want around, right? So you’re saying if you favorite these or pin them, in our words here, you have a pin section at the top. But if you start making a lot of chats, the non-pin ones will get.Esteban: second.Gabriel: kind of blown away first.Esteban: Exactly. Exactly. ⁓ We have plenty of requests about chat organization and some people are going as far as like, want to create projects, want to create groups, I want to organize my stuff, which it all makes total sense. And I guess up to a point you need those sort of organization tools, but the simple ones will cover most of the needs for most of our users, I would say, because maybe you have a few chats where you keep coming back to them, but then you have a bunch of quick requests, quick checkups with the LLM.And something like this is super simple. It doesn’t require a lot of effort. And yeah, we hope it helps a lot of our users.Gabriel: And you mentioned in there a few times like we did this and we’re working on this, these features in particular because we get a lot of user requests for them. ⁓ So speaking to that a little bit, like my understanding is, you know, we get lots of feedback. ⁓ We’re looking through it all and kind of organizing it, like which are the most like important issues to work on. But then also when we launch features, cause to your point, people ask for all sorts of different things. And then we, and like you in particular, design. And you’re like, well, I think this is gonna be a satisfaction of a lot of people’s requests, even if that’s not exactly the thing they asked for. And then we put it out and then how do we know whether it worked or not? Like, what are we looking at?Esteban: Yes, so to answer your question, we see a usage in particular. We don’t have, of course, data about a particular users, but we know that roughly X amount of people are using this feature. We also know from social media, we got a little bit of love ⁓ after posting that we launched this. And then the other thing that’s a really interesting metric is we see the feedback coming in. ⁓ and comparing with what kind of feedback we were getting a month ago, and we see a big decrease on charge organization requests. As said, there are still things that we can do and we’re working on them, but we see a lot of people, a lot less people requesting for things like this, which is also good news.Gabriel: I love that metric. mean, because it really is, I guess we are lucky that we have enough users and enough sample size where we can be like, wow, chat organization is a category we can ascribe feedback to. And now it’s like halved or whatever after these features come out, right?Esteban: Right. Yeah, so something we did for this project that is super interesting, it was fun for me to do was that we have this category, like chat organization, and then we get feedback directly from our users asking for a specific feature. I went and looked into all the feedback that was related to chat organization, and I tried to see why we’re asking for a specific feature. Some of them actually mentioned it, no, I want this so I can do that. And then those needs, I matched them with what type of features will work, even just by naming different types of features. You said that this was sort of favorites or it could be pinning or it could be saved or it could be bookmarked. We also have bookmarks in the browser. So how do we name the feature related to the benefit that I will have and the need that it will solve, but also how much effort will require from the users to actually get the benefit that they were asking for? ⁓ Create projects, it’s a lot of effort.And they will get a lot of benefit, but not many people are willing to spend time organizing on their chats. But pinning is one tap away So that’s a sort of ⁓ prioritization exercise that we did to define exactly what are the new features that we’re going to launch.Gabriel: Yeah, that’s a super interesting point because like all different features have different levels of complexity for us to build. And then also for the user to actually use to your point, like I’ve seen, yeah, I’ve seen lots of requests for people wanting really complex things, but then very small percentage of people would actually go through the effort to use those things. Whereas to your point here, pinning, especially the chat title is just automatic. We get that for free to everybody. But then the pinning is just literally just a one click. You don’t have to type in a name for it, a folder name for example. You just get the pinning at the top.Esteban: Right. And in some cases you actually need to go and give effort to the users in a way, because we thought automatic title would solve the problem. No, like we just add automatic title to everything. That’s it. And then by testing and seeing how some of the titles reacted, we knew that we were, it’s just not going to be bulletproof. Like in some cases you want to have your own title. So we went back and said, okay, let’s add a manual option. ⁓ Cause then we went, then that way we were solving everyone’s needs. Yeah.Gabriel: You’re like covered. Yeah, you’re covered every, at least there’s a backstop. That’s super interesting. So what’s next, if anything? Are you moving on from chat organization at the moment or are you thinking about some other kind of deeper aspects of it?Esteban: It’s still there and it’s still one of our top requests. You mentioned that we’re working on finding ways to give you a bigger list of like bigger limits, have more chats, maybe syncing between devices. And we are also thinking about search, of course, ⁓ so you don’t have to read your whole list of chats, just go straight into that answer that you remember. ⁓ from a week ago. So those are things we’re working on and I’m sure there will be plenty more organization features we could do but for the moment that’s a plan.Gabriel: Sweet. Well, thank you. I’m excited to have you back and someone, maybe I hope I’m the host, but somebody to talk about these more organization features in the future. But yeah, thanks for coming on and thanks everybody for listening. Bye everybody.Esteban: Okay. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit insideduckduckgo.substack.com