Very large context windows, SearchGPT, AI agent. What are they?

Important things I learned from leaked Stanford interview with former Google CEO, Eric Schmidt.

A few weeks ago, the former Google CEO, Eric Schmidt, gave a very informative and controversial talk about AI to Stanford students. While Schmidt thought everything he said was kept confidential, the talk was being streamed live. Very soon, the video was taken down (but, of course, people made backups).

As a designer who is growing my career alongside AI innovations, I find this video offers insider’s knowledge to direct my attention in the noisy world of AI development. 

I will share my thoughts on a few key concepts from the talk that stood out to me. Let me know if anything was explained factually incorrect.

---

Technology: Very large context windows will be available very soon.

Very long context windows for large language models (LLMs) will be available in the next 1-2 years, and it will unlock new functionalities of the LLM applications we use today. This is very relevant to designers. I will elaborate with use cases in the next section about applications.

What is a context window? For example, when a friend asks for my help to plan a trip to New York, I would already know the activities they are into, how much they are willing to spend, and dietary preferences based on all the previous conversations in our friendship. This background knowledge enables me to give them relevant, non-repetitive, and personalized information about the trip. 

Context window is the memory to keep such background knowledge in mind for machines. The more information the LLM can remember from previous conversations, the more relevant and accurate its responses are going to be. The context window includes both the text in the user-generated questions and the LLM-generated responses. Having larger context windows means the LLM applications will be able to answer questions referencing past conversations from many many chats ago, as well as from multiple books and documents the user feeds to them.

Up until a few days ago, Google’s Gemini had the longest context window with 1 million tokens, which is the equivalent of 1500 pages of text. On average, a book only has 200-350 pages. However, the new model that broke Google’s record has 100 million tokens, the equivalent of more than 400 books. This happened 6 months after Google announced the 1 million tokens context window this February. As Eric Schmidt pointed out in his talk (before this new model came out), AI development is going so rapidly that he may need to give a new talk to debrief the changes every 6 months.

As designers, we must pay attention. Technological advancements generate new opportunities to solve existing problems.

Applications: AI agents will use SearchGPT to provide us with the most up to date information from the web.

Large context windows makes it possible for AI to crawl the web to read many web pages and long documents in order to provide the most up to date answers to a user’s question. This is what SearchGPT does as an AI-powered search engine (read further on this very detailed explanation differentiating SearchGPT from Google search). It summarizes the most up-to-date information from the web as answers to user’s questions, while providing reference web links as the source of the answers. When would SearchGPT be useful? I think the most obvious answer is when acquiring time-sensitive information is crucial, and the information must be accurate.

Example 1:

In the early months of the COVID-19 pandemic in 2020, the world was shocked by the virus’ impact, and was too unprepared to adjust. I constantly read news articles and listened to podcasts to follow the evolving understanding about how the virus spreads, effective precautions methods, vaccination developments, and infected and hospitalized populations. On some days, hours easily slip into learning about how this world was falling apart. If a tool like SearchGPT existed, I probably could have learned the most up to date COVID-19 knowledge from clearly and concisely summarized descriptions from the latest research publications, along with web links as references, rather than spending several depressing hours trying to make sense of various media interpretations on my own. 

Example 2:

Here is another example. Along with AI agents, SearchGPT could also be very useful in planning the New York trip mentioned earlier. AI agents can gather user requests, analyze the information SearchGPT found, then present the information to you in a way that’s easy to comprehend. 

Summer is an expensive time to visit New York. SeachGPT could get you the most up to date deals on hotels, flights, and restaurants. How is it different from just doing a Google search? Rather than searching for hotels, flights, and restaurants individually, you could send a query to an AI agent requesting an itinerary based on the activities you are into, how much you are willing to spend, dietary preferences, and neighborhood you’d like to stay in. The AI agent would use SearchGPT to find the information that meet your requirements, then present it to you in a way that’s easy to scan for the days, and corresponding activities and costs.

AI agent is defined at Google IO 2024 by current Google CEO, Sundar Pichai, as “intelligent systems that show reasoning, planning, and memory. They are able to “think” multiple steps ahead, and work across software and systems, all to get something done on your behalf, and most importantly, under your supervision.” See source.

On top of it, you could ask the agent to notify you when there is a new deal on flights lower than the originally proposed one. The AI agent would use SearchGPT to find the latest flight information that matches the dates and time you requested. 

The problem is that the two new flights AI agent found are by Delta Airline. AI agent remembers you complained about Delta Airlines' significant delay a year ago. To solve this problem, it asks if you want to know the percentages of delayed Delta flights in the past 3 months. Then, it would use SearchGPT to gather the information, analyze it, then present the information along with the pros and cons of taking Delta at this affordable price.

Job: Work life balance does not help companies win the AI race

This statement by Eric Schmidt in the talk received many controversies. Even though Google invented the transformer architecture (i.e., the “T” in GPT stands for “Transformer”), Schmidt thinks the company fell behind on AI (for now) because they prioritize employees’ work life balance. The rationale is that companies whose employees are highly motivated and prioritize work will deliver better results faster than those that do not. 

I agree with this rationale, but it becomes problematic when one’s wants and needs in life conflict with commitments at work. The good news is that I also think we can live with this conflict most of the time when the stake is not high. I believe cultivating a fulfilling life is as important as an accomplished career. For some people, life fulfillment has a bigger overlap with career accomplishment. That’s okay, but there are still relationships outside our career that are the pillars of our happiness. It’s a process to learn to constantly give them time and attention to keep them sustained and flourish. 

In Schmidt’s talk, he discussed work life balance and remote work in the context of Stanford graduates founding their own companies in the future. He asked students if they would want their employees to come to the office once a week while competing startups are not. If I switch to the founder’s mindset, where the founder knows they would take the direct shock of the company’s downfall, I would be more nervous if the employees are relaxed about delivering results than how many times they come to office. From the employees’ viewpoint, when important personal relationships directly impact my life fulfillment, I would be defending my time and mental space against work demands so that I’m capable of cultivating healthy relationships. If the conflicts between life and work become non-negotiable, then it reveals the knowledge that the environment I chose for work no longer fits the current phase of my life. 

To sum it up, I believe what Eric Schmidt said is generally true, and works well to reach success. Use it as a mirror to find out if that kind of life is for us across our career. No need to hate on it.

Anecdotally, the few people I know working in design at Google actually have demanding roles. However, when life circumstances arise, Google’s flexible work policy makes spending a generous amount of time addressing those personal circumstances possible, and it is appreciated. I’d like to believe this appreciation and trust can fuel productive work and retain talented people for longer. 

Lastly, if you want to know what it’s like working at Google, reach out to designers at Google on LinkedIn, and ask them yourself. In case you don’t know yet, here’s how you can find designers at Google: 

1. Find Google’s LinkedIn page

2. Navigate to the People tab

3. Type in “product designer” in the search bar

4. Keep scrolling down to see the list of designers

---