Pete LePage

Super excited to share that we just launched 𝗥𝗲𝗿𝗮𝗻𝗸𝗲𝗿 for 𝗩𝗲𝗰𝘁𝗼𝗿 𝗦𝗲𝗮𝗿𝗰𝗵 — making RAG agents even smarter! 🚀 RAG adoption continues to grow, and what really matters is end-to-end quality. But simply giving an LLM more documents isn’t always the answer. Even though large-context models exist, they can still fall short, since quality isn’t guaranteed just by adding context — something I wrote about earlier (https://lnkd.in/g7jJ3mfr). What makes the difference is better ranking that brings the most relevant context to the top when it counts. Building on that, in our paper — 𝗗𝗿𝗼𝘄𝗻𝗶𝗻𝗴 𝗶𝗻 𝗗𝗼𝗰𝘂𝗺𝗲𝗻𝘁𝘀: 𝗖𝗼𝗻𝘀𝗲𝗾𝘂𝗲𝗻𝗰𝗲𝘀 𝗼𝗳 𝗦𝗰𝗮𝗹𝗶𝗻𝗴 𝗥𝗲𝗿𝗮𝗻𝗸𝗲𝗿 𝗜𝗻𝗳𝗲𝗿𝗲𝗻𝗰𝗲 — we found that adding more documents boosts quality at first, but beyond a certain point diminishing returns kick in and quality can even drop because of “phantom hits” (irrelevant passages scored too highly). In other words, quality beats quantity — you want to prioritize the most relevant passages, not just more context. Link in the comments. That’s exactly where 𝗩𝗲𝗰𝘁𝗼𝗿 𝗦𝗲𝗮𝗿𝗰𝗵 𝗥𝗲𝗿𝗮𝗻𝗸𝗲𝗿 comes in. By adding just one parameter to your Vector Search query, you unlock a big quality boost: 𝟴𝟵% of the time the correct answer shows up in the top‑10, a +𝟭𝟱 𝗽𝗼𝗶𝗻𝘁 lift over baseline (74%) and +𝟭𝟬 𝗽𝗼𝗶𝗻𝘁𝘀 over leading cloud alternatives. Really proud of this A+ team that not only built the reranker and retriever at scale, but also co‑authored the paper together with our amazing Databricks Mosaic Research team. Couldn’t ask for a sharper crew! 💪 You can be part of this team too — 𝘄𝗲’𝗿𝗲 𝗵𝗶𝗿𝗶𝗻𝗴 𝗲𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝘀 𝗮𝗻𝗱 𝗺𝗮𝗻𝗮𝗴𝗲𝗿𝘀 𝗮𝘁 𝗮𝗹𝗹 𝗹𝗲𝘃𝗲𝗹𝘀 to work on AI products, billion‑scale retrieval systems, and high‑load serving infra. If that excites you, let’s talk! 👤 Senior Engineering Manager https://lnkd.in/g-Nvy64X 👤 Senior Staff Software Engineer https://lnkd.in/g8Yhwx2t 👤 Staff Software Engineer https://lnkd.in/gpHx-nrR 👤 Senior Software Engineer https://lnkd.in/gzFTwnTc

546

18 Comments

So Jensen Huang just told reporters he “pretty much disagrees with almost everything” Dario Amodei says about AI. That’s quite the statement :-). Here’s what went down at VivaTech: Dario Amodei, CEO of Anthropic, warned that AI could wipe out 50% of entry-level white-collar jobs in five years. His message was--Governments need to stop pretending everything’s fine. Jensen Huang’s response? “Nah.” (Paraphrasing: “He thinks AI is so dangerous that only his company should be allowed to build it.”) Meanwhile, Yann LeCun is over here calmly comparing it all to the printing press revolution. But here’s what’s been bothering me about this whole debate: We’re so busy arguing about whether AI will take our jobs, we’re ignoring a deeper question: What are we actually doing that a machine shouldn’t do? Seriously--if your entire job can be automated, maybe that’s not where human brilliance should be spent in the first place. Last week, I watched my wife help our son through a math meltdown. She didn’t just fix the problem--she saw the fear behind it. She soothed, encouraged, made him laugh, then helped him believe he could do it. Try automating that. Or my friend who’s a startup founder. Sure, AI helps him code faster. But his real job? Convincing skeptical investors. Rallying exhausted engineers. Making gut calls that go against the data--but turn out to be right. The irony? Huang actually said it best: “You won’t lose your job to AI. You’ll lose it to someone using AI.” So maybe the question isn’t will AI replace us. Maybe it’s what parts of our work are even worth keeping? Because if you are being honest--so much of our workday already feels robotic: Filling out forms. Copy-pasting reports. Writing emails that say nothing. Let the machines do that. I want to keep the messy, beautiful, deeply human parts of work: The mentoring. The “Hey, are you okay?” conversations. The creative leaps from nowhere. The ethical decisions that require context no machine can hold. What’s the most human thing you do in your job that you’d never want to automate? #AI #FutureOfWork #RealTalk #WorkThatMatters

40

2 Comments

For researchers looking to explore how AI systems can better understand the world from a human perspective, we are introducing Aria Gen 2, the latest research glasses from Meta’s Project Aria. With upgraded sensors, on-device machine perception, and real-time audio interactions, Aria Gen 2 unlocks new possibilities in egocentric machine perception, contextual AI, and robotics, from new datasets to real-time prototyping of contextual AI experiences. Details here: https://lnkd.in/gtmDPYj3

54

1 Comment

Platform engineering is the future 🚀 Developers are at the core of everything we do at PagerDuty 💚 Our latest collaboration with Spotify for Backstage brings PagerDuty’s industry-leading Incident Management directly into Backstage, the open platform reimagining developer experience. With the new PagerDuty Plugin for Backstage, devs can see incidents, on-call context, and service ownership right where they already work : Less dilution ; More focus. Let’s go 🎉 Read more 👇 https://lnkd.in/gnsq8Nr9

28

2 Comments