1 00:00:10,720 --> 00:00:14,015 Mike Rugnetta: Friends, hello, and welcome to Never Post, a 2 00:00:14,015 --> 00:00:16,255 podcast for and about the Internet. I'm your host Mike 3 00:00:16,255 --> 00:00:21,215 Rugnetta. This intro was written on Tuesday, 06/23/2026 at 4 00:00:21,215 --> 00:00:28,255 10:35AM eastern, and we have a fantasmagoric show for you this 5 00:00:28,255 --> 00:00:31,410 week. In this episode, Audrey chats with assistant professor 6 00:00:31,410 --> 00:00:34,530 at the University of Maryland College of Information, Mauls 7 00:00:34,530 --> 00:00:38,370 Souder, and writer, researcher, and community organizer, Erin 8 00:00:38,370 --> 00:00:43,695 Kissane, about the incursion of AI models hallucinations into 9 00:00:43,695 --> 00:00:47,695 increasingly prevalent corners of media, law, government, and 10 00:00:47,695 --> 00:00:51,615 culture, and what this means overall for the production of 11 00:00:51,615 --> 00:00:55,055 knowledge. But first, we're gonna take a quick break. 12 00:00:55,055 --> 00:00:57,295 You're gonna listen to some ads unless you're on the member 13 00:00:57,295 --> 00:01:00,150 feed. And when we return, we're gonna talk about a few of the 14 00:01:00,150 --> 00:01:03,110 things that have happened since the last time you heard from us. 15 00:01:04,150 --> 00:01:08,150 Zephyr in the sky at night, I wonder. Do my tears of morning 16 00:01:08,150 --> 00:01:12,790 sink beneath the sun? She's got herself a universe gone quickly 17 00:01:12,935 --> 00:01:15,255 for the call of thunder threatens. 18 00:01:15,815 --> 00:01:25,095 Six stories this week. France says to Palantir. French PM 19 00:01:25,095 --> 00:01:27,415 Sebastian Lecourneau announced last week that French 20 00:01:27,415 --> 00:01:31,140 intelligence agency, the general directorate for internal 21 00:01:31,140 --> 00:01:34,820 security, would be cutting ties with the techno fascism purveyor 22 00:01:34,820 --> 00:01:40,420 in order to reduce reliance on American technology. Lecourneau 23 00:01:40,420 --> 00:01:43,860 says the gap will in time be filled by French AI developer, 24 00:01:43,860 --> 00:01:47,715 CHAP's vision. This comes as England's NHS, the National 25 00:01:47,715 --> 00:01:51,395 Health Service, considers ditching its Palantir contract 26 00:01:51,395 --> 00:01:55,155 under pressure from MPs and workers alike over privacy and 27 00:01:55,155 --> 00:01:56,435 security concerns. 28 00:01:56,995 --> 00:02:00,590 Now I don't wanna speak too soon here, but I'm starting to think 29 00:02:00,590 --> 00:02:06,110 that maybe Palantir is bad. 30 00:02:08,270 --> 00:02:08,830 Mols Sauter: No. 31 00:02:09,710 --> 00:02:12,350 Mike Rugnetta: China has been increasing its global telecom 32 00:02:12,350 --> 00:02:15,845 presence in South America. The most recent partnership offering 33 00:02:15,845 --> 00:02:20,245 being made to Chile. State owned China Mobile offered to 34 00:02:20,245 --> 00:02:23,605 construct an undersea cable linking the two nations. But in 35 00:02:23,605 --> 00:02:27,410 February of this year, rest of world reports, The US stepped in 36 00:02:27,410 --> 00:02:30,530 and sanctioned the South American nation, revoking the 37 00:02:30,530 --> 00:02:35,090 visas of three officials. This after a similar 2019 scheme was 38 00:02:35,090 --> 00:02:37,730 kiboshed also due to US pressure. 39 00:02:37,890 --> 00:02:40,050 The state department has said the infrastructure projects 40 00:02:40,050 --> 00:02:43,065 would have, quote, compromised critical telecommunications 41 00:02:43,065 --> 00:02:46,505 infrastructure and undermined regional security in our 42 00:02:46,505 --> 00:02:50,505 hemisphere, end quote. The US and China remain engaged in an 43 00:02:50,505 --> 00:02:53,545 ongoing data center and AI development arms race that 44 00:02:53,545 --> 00:02:59,500 pretty much no one has asked for or benefits from. We. Music, all 45 00:02:59,500 --> 00:03:03,180 of it has been ingested by AI. Writing for the Atlantic, Alex 46 00:03:03,180 --> 00:03:06,860 Reisner reveals that he was able to get a hold of four training 47 00:03:06,860 --> 00:03:09,340 datasets to give a sense of their size. 48 00:03:09,340 --> 00:03:13,185 Only one is comprised of 12,000,000 tracks that would 49 00:03:13,185 --> 00:03:18,545 take 91 to listen through. AI developers and researchers 50 00:03:18,545 --> 00:03:21,745 employ these datasets to develop their products, though some, 51 00:03:21,745 --> 00:03:24,840 like Suno, say there are guardrails in the technology to 52 00:03:24,840 --> 00:03:28,120 prevent against the models generating faithful or near 53 00:03:28,120 --> 00:03:32,120 copies of existing music. Reisner writes quote, what the 54 00:03:32,120 --> 00:03:36,200 datasets illustrate primarily is the scale and variety of music 55 00:03:36,200 --> 00:03:39,855 easily available to AI developers. Companies often 56 00:03:39,855 --> 00:03:44,015 claim to use only content that is freely available online, but 57 00:03:44,015 --> 00:03:47,135 the datasets reveal the quantity of downloadable music that 58 00:03:47,135 --> 00:03:50,495 developers can access even though it is not supposed to be 59 00:03:50,495 --> 00:03:54,310 free, end quote. The dataset is searchable if you wanna go see 60 00:03:54,310 --> 00:03:57,830 if your work has been gobbled up by the endless ravenous maw of 61 00:03:57,830 --> 00:03:58,870 Silicon Valley. 62 00:03:59,990 --> 00:04:04,230 YouTube has settled a Florida case in which a teenager named r 63 00:04:04,230 --> 00:04:08,225 k c in court documents alleges the platform was designed to be 64 00:04:08,225 --> 00:04:12,225 addictive leading to anxiety, sleep deprivation, and other ill 65 00:04:12,225 --> 00:04:16,625 effects. R k c is also suing Meta, TikTok, and Snap in 66 00:04:16,625 --> 00:04:20,465 California according to the BBC. This comes after a string of 67 00:04:20,465 --> 00:04:24,600 various settlements and findings in users favor regarding the 68 00:04:24,600 --> 00:04:27,560 deleterious effects of features like infinite scroll, 69 00:04:27,560 --> 00:04:31,640 algorithmic recommendation, and so on. Most notably, six months 70 00:04:31,640 --> 00:04:37,335 ago, Australia banned teens from using social media entirely 71 00:04:37,575 --> 00:04:41,495 countrywide, a massive undertaking which researchers in 72 00:04:41,495 --> 00:04:45,015 the British Medical Journal this month note has been particularly 73 00:04:45,495 --> 00:04:50,135 ineffective writing, quote, we found insufficient evidence to 74 00:04:50,135 --> 00:04:53,880 conclude that exposure to the act had any early substantial 75 00:04:53,880 --> 00:04:59,400 effects on social media use among adolescents aged 16 years. 76 00:04:59,800 --> 00:05:03,160 The findings contrast with similar legislative requirements 77 00:05:03,160 --> 00:05:04,440 of web based services. 78 00:05:04,795 --> 00:05:08,715 For example, in The UK where access to pornographic websites 79 00:05:08,715 --> 00:05:13,995 substantially reduced by 30 to 50% following the introduction 80 00:05:13,995 --> 00:05:20,315 of age assurance measures, end quote. Whoops. Oh, well, I 81 00:05:20,315 --> 00:05:25,060 guess. Oops. Mister Rogers is on YouTube. 82 00:05:25,060 --> 00:05:27,540 Okay. Maybe it's not so bad out there after all. The channel 83 00:05:27,540 --> 00:05:30,420 will feature clips as well as full episodes many of which have 84 00:05:30,420 --> 00:05:34,035 not aired or been otherwise available for quite some time. 85 00:05:34,115 --> 00:05:36,835 The Fred Rogers Foundation has partnered with British digital 86 00:05:36,835 --> 00:05:41,635 content agency, Little Dot Studios, which as of 2020 87 00:05:41,635 --> 00:05:47,395 managed over 600 branded social video channels for various 88 00:05:47,395 --> 00:05:49,880 clients. Won't you please? 89 00:05:50,440 --> 00:05:58,520 Won't you please? Please won't you be my neighbor? My neighbor? 90 00:05:59,000 --> 00:06:04,425 I'm glad we're together again. And finally, are you seeing what 91 00:06:04,425 --> 00:06:05,225 I'm seeing? 92 00:06:05,785 --> 00:06:09,145 Hallucinations in the news. At their recent IO conference, 93 00:06:09,145 --> 00:06:12,345 Google unveiled their redesigned search product in which results 94 00:06:12,345 --> 00:06:16,640 are delivered not as a list of links, but a chat with an LLM. 95 00:06:16,960 --> 00:06:19,280 Searchers have found this experience increasingly 96 00:06:19,280 --> 00:06:23,280 prevalent as the company has rolled out its AI overviews. 97 00:06:23,440 --> 00:06:26,800 Summaries of search results aimed at synthesizing available 98 00:06:26,800 --> 00:06:30,225 information without requiring additional clicks. But there's 99 00:06:30,225 --> 00:06:31,105 one problem. 100 00:06:31,665 --> 00:06:35,505 Well, there are actually many problems, but one is the well 101 00:06:35,505 --> 00:06:40,225 documented prevalence of overview hallucinations, which 102 00:06:40,225 --> 00:06:45,160 recently a German court has ruled Google is liable for. The 103 00:06:45,160 --> 00:06:49,000 ruling describes the overviews as quote, the defendant's own 104 00:06:49,000 --> 00:06:53,080 statements. It alone has influence over the AI's offering 105 00:06:53,080 --> 00:06:56,840 and the algorithms with which the AI operates, end quote. 106 00:06:57,275 --> 00:07:00,795 Google's got good company in its hallucinatory woes. Number 107 00:07:00,795 --> 00:07:04,395 Cruncher's KPMG had to recently retract a report titled of all 108 00:07:04,395 --> 00:07:08,555 things redefining excellence in the age of AI because it was 109 00:07:08,555 --> 00:07:10,635 riddled with errors and inventions. 110 00:07:12,330 --> 00:07:15,930 Ernst and Young recently retracted a report on loyalty 111 00:07:15,930 --> 00:07:19,530 programs because it contained hallucinations. Wikipedia has 112 00:07:19,530 --> 00:07:23,610 unsurprisingly had to ban AI contributions because they are 113 00:07:23,610 --> 00:07:26,565 often factually incorrect. And it feels like every week there's 114 00:07:26,565 --> 00:07:29,765 a new story about a law firm caught using AI after citing 115 00:07:29,765 --> 00:07:36,245 some nonexistent precedent. What is going on? How do we 116 00:07:36,245 --> 00:07:40,965 understand and protect ourselves against the incursion of opaque 117 00:07:41,070 --> 00:07:44,670 machine invention into what have otherwise been the most 118 00:07:44,830 --> 00:07:49,950 calculatory, document dependent, and knowledge based industries. 119 00:07:50,270 --> 00:07:54,965 And what if these hallucinations are not a bug, but a feature? A 120 00:07:54,965 --> 00:07:58,885 fundamental aspect of the technology revealing in some way 121 00:07:59,125 --> 00:08:00,645 what it really is. 122 00:08:00,885 --> 00:08:01,685 Audrey Evans: Uh-huh. 123 00:08:01,765 --> 00:08:03,525 Mike Rugnetta: We're gonna take a quick break, and when we 124 00:08:03,525 --> 00:08:07,365 return, Audrey talks with Malz Sauter, assistant professor at 125 00:08:07,365 --> 00:08:10,210 the University of Maryland, where they teach the history and 126 00:08:10,210 --> 00:08:14,210 philosophy of technology, and Aaron Kissane, a writer and 127 00:08:14,210 --> 00:08:18,050 researcher focused on knowledge and networks at unbreaking.org 128 00:08:18,050 --> 00:08:21,845 and previously, the COVID tracking project. They'll all 129 00:08:21,845 --> 00:08:25,925 talk about what AI hallucinations actually are from 130 00:08:25,925 --> 00:08:28,645 a knowledge production standpoint and how the 131 00:08:28,645 --> 00:08:31,845 information they contain reflects a much longer and 132 00:08:31,845 --> 00:08:35,685 stranger facet of human culture than you may have previously 133 00:08:35,685 --> 00:08:36,165 considered. 134 00:08:48,435 --> 00:08:51,075 Audrey Evans: In May, Google announced that they are turning 135 00:08:51,075 --> 00:08:55,635 from legacy search, the search box, to AI search as the default 136 00:08:55,635 --> 00:09:00,355 search mode. A May 16 TechCrunch article even had the headline, 137 00:09:00,890 --> 00:09:05,690 Google search as you know it, it's over. The ground of the web 138 00:09:05,690 --> 00:09:09,850 is shifting. The very way we step into our interactions with 139 00:09:09,850 --> 00:09:13,930 the web is changing quickly. Google suggested this as a quote 140 00:09:13,930 --> 00:09:20,285 paradigm shift, moving users from search into generative UI 141 00:09:20,765 --> 00:09:21,645 experiences. 142 00:09:22,445 --> 00:09:25,405 So instead of returning links, Google search will now be 143 00:09:25,405 --> 00:09:29,805 dropping users into quote AI powered interactive experiences 144 00:09:30,000 --> 00:09:34,720 and be dispatching information agents to gather information on 145 00:09:34,720 --> 00:09:38,960 a user's behalf. So search will now look more like a long back 146 00:09:38,960 --> 00:09:43,905 and forth conversation where the user will also be suggested 147 00:09:43,905 --> 00:09:49,265 prompts, directions. This is really dropping users into an 148 00:09:49,265 --> 00:09:53,985 entirely new research process experience. One even further 149 00:09:53,985 --> 00:09:56,945 intermediated by Google and driven by the incentives and 150 00:09:56,945 --> 00:10:00,400 priorities of this company. This paradigm shift that they're 151 00:10:00,400 --> 00:10:03,840 talking about is one that's now less about information retrieval 152 00:10:03,840 --> 00:10:08,240 as a process and now something that's geared towards prompting 153 00:10:08,240 --> 00:10:13,125 users to quote, take actions and actions that are going to be 154 00:10:13,125 --> 00:10:15,365 defined by their models. 155 00:10:16,405 --> 00:10:20,325 I know both of you study the way we collectively use and 156 00:10:20,325 --> 00:10:23,765 synthesize information, how we turn information to action, 157 00:10:24,660 --> 00:10:29,140 sometimes collective action in our society. So I wanted to get 158 00:10:29,140 --> 00:10:33,860 your reactions. And Erin, it is such a delight to be here with 159 00:10:33,860 --> 00:10:35,940 you today. Mols, let's start with you. 160 00:10:40,085 --> 00:10:42,165 Mols Sauter: This is a really interesting question because 161 00:10:42,165 --> 00:10:45,685 there are a bunch of different things that are going on both 162 00:10:45,685 --> 00:10:49,045 inside sort of what we'll call sort of like the AI this current 163 00:10:49,045 --> 00:10:53,125 AI moment, but also these different specific systems. So, 164 00:10:53,125 --> 00:10:58,170 like, let's just take the search bar. And I love that you 165 00:10:58,170 --> 00:11:01,850 specifically flagged the paradigm shift. Whenever someone 166 00:11:01,850 --> 00:11:04,330 says paradigm shift, I'm like, cool. Now we have to talk about 167 00:11:04,330 --> 00:11:07,370 what a paradigm shift actually is and what it what it means to 168 00:11:07,370 --> 00:11:08,810 have a paradigm shift. 169 00:11:09,050 --> 00:11:13,645 And so what we can think about it as when did search stop being 170 00:11:14,125 --> 00:11:19,965 we get returned results that are pointers to things that exist, 171 00:11:19,965 --> 00:11:24,380 that people have written and they've put on the Internet as 172 00:11:24,380 --> 00:11:30,540 objects, and we find them via object identifiers. Two, we get 173 00:11:30,540 --> 00:11:35,260 a natural language response from a technical system that also 174 00:11:35,260 --> 00:11:39,365 points to sometimes these objects. It has some 175 00:11:39,365 --> 00:11:42,565 relationship to these objects, but it is not the object. It's 176 00:11:42,565 --> 00:11:44,965 not merely a referent. The point of the search bar is not to 177 00:11:44,965 --> 00:11:45,365 direct. 178 00:11:45,365 --> 00:11:48,005 The point of the search bar is to answer. So how do we get this 179 00:11:48,005 --> 00:11:51,765 paradigm shift? And one of the reasons I think that it doesn't 180 00:11:51,765 --> 00:11:54,860 feel like a paradigm shift to us, the reason why, like, for 181 00:11:54,860 --> 00:11:58,460 the users, we're like, wait a minute. We're not advancing. 182 00:11:58,460 --> 00:11:59,420 This isn't an advancement. 183 00:11:59,420 --> 00:12:02,060 This isn't better. I'm not happy about what is happening right 184 00:12:02,060 --> 00:12:05,900 now. I don't feel like my experience has become more, is 185 00:12:06,325 --> 00:12:09,525 it's not actually following what causes paradigm shifts. So one 186 00:12:09,525 --> 00:12:12,565 of the key things in a paradigm shift in scientific theory is 187 00:12:12,565 --> 00:12:17,125 the crisis and anomalies stage, which is when you've got the 188 00:12:17,125 --> 00:12:21,220 theory, it is not paradigmatic yet, you are collecting your 189 00:12:21,220 --> 00:12:23,940 experimental results, you're building your literature, you're 190 00:12:23,940 --> 00:12:28,100 building your advocates, you're you're assembling your team, and 191 00:12:28,100 --> 00:12:30,500 then you enter into that period of norm what we call normal 192 00:12:30,645 --> 00:12:32,885 science, which is when we've accepted the paradigm. Now we're 193 00:12:32,885 --> 00:12:34,565 just sort of like cleaning up around the edges. 194 00:12:34,565 --> 00:12:37,765 We're puzzle solving. Maybe it's an even entering into an 195 00:12:37,765 --> 00:12:39,925 engineering mode where we're just sort of like, yeah, this is 196 00:12:39,925 --> 00:12:42,325 a thing that solves problems. We use it to solve problems. We're 197 00:12:42,325 --> 00:12:44,405 not necessarily discovering anything new in the normal 198 00:12:44,405 --> 00:12:48,350 science mode. But then we enter into crisis and anomaly stage 199 00:12:48,350 --> 00:12:49,390 when things stop working. 200 00:12:49,390 --> 00:12:52,830 We start building up these little inconsistencies, these 201 00:12:52,830 --> 00:12:55,550 little holes, and they start accumulating to the point where 202 00:12:55,550 --> 00:12:58,990 the the theory or the paradigm can no longer sustain itself, 203 00:12:58,990 --> 00:13:02,005 and it enters into a period of crisis. And that's when things 204 00:13:02,005 --> 00:13:05,765 open up for a new paradigm. That's basically the structure 205 00:13:05,765 --> 00:13:09,525 of, like, how we how do we change these big ideas? And one 206 00:13:09,525 --> 00:13:13,125 of the things that is currently happening is the old ideas have 207 00:13:13,125 --> 00:13:17,330 never arrived at the crisis and anomaly stage. The idea that a 208 00:13:17,330 --> 00:13:21,890 search bar points you to resources was perfectly happy 209 00:13:21,890 --> 00:13:25,010 chilling out in the puzzle solving, like like, it's not 210 00:13:25,010 --> 00:13:27,490 science, but quote unquote normal science space where it's 211 00:13:27,490 --> 00:13:28,850 like, this is good. 212 00:13:28,850 --> 00:13:31,815 This works. We're happy. We don't have lots of problems. 213 00:13:31,815 --> 00:13:34,295 Like, there are some problems, but they're not fatal problems. 214 00:13:34,295 --> 00:13:35,975 And they're certainly not fatal problems with the basic 215 00:13:35,975 --> 00:13:36,775 mechanism. 216 00:13:37,015 --> 00:13:40,295 Instead, we have sort of this forced paradigm shift where a 217 00:13:40,295 --> 00:13:44,700 bunch of companies, and by a bunch, mean, like, four, are 218 00:13:44,780 --> 00:13:48,540 pushing an entirely new way of both interacting with the web, 219 00:13:48,540 --> 00:13:52,140 interacting with certain types of knowledge, and interacting 220 00:13:52,140 --> 00:13:56,215 with certain types of media. And everyone's reaction is quite 221 00:13:56,215 --> 00:13:59,735 expected where it's like, wait a minute. Wait a minute. We 222 00:13:59,735 --> 00:14:01,415 weren't ready. Like, we're not ready to do this. 223 00:14:01,415 --> 00:14:04,215 There was nothing wrong with that in a fatal way. 224 00:14:04,215 --> 00:14:08,070 Erin Kissane: I think it's it's impossible to overemphasize how 225 00:14:08,070 --> 00:14:10,790 hard some of these tech companies have been trying to 226 00:14:10,790 --> 00:14:15,270 make paradigm shift happen for years now. So, obviously, this 227 00:14:15,270 --> 00:14:19,750 is in continuity with trying to make metaverse happen, trying to 228 00:14:19,750 --> 00:14:24,755 make NFTs happen, trying to make crypto happen in specific ways. 229 00:14:24,755 --> 00:14:28,115 Not that any of those technologies did not exist, but 230 00:14:28,115 --> 00:14:31,395 none of them existed in the ways that the tech companies pushing 231 00:14:31,395 --> 00:14:36,440 them were presenting them to us. So especially given that a lot 232 00:14:36,440 --> 00:14:39,640 of the folks leading these technology companies were around 233 00:14:39,640 --> 00:14:44,200 for what I would classify as a genuine paradigm shift, the 234 00:14:44,200 --> 00:14:48,925 arrival of search, the arrival of what used to be the open web 235 00:14:48,925 --> 00:14:53,245 as a pretty substantial shift in the way that we interact with 236 00:14:53,245 --> 00:14:58,205 knowledge as societies. That actually happened and they 237 00:14:58,205 --> 00:15:03,260 figured out gradually how to enclose more and more of it, but 238 00:15:03,260 --> 00:15:05,020 then they kind of hit a wall. 239 00:15:05,500 --> 00:15:09,180 I don't want to be fair to anyone, but to be fair, I will 240 00:15:09,180 --> 00:15:14,380 say the arms race of search has led to, I think many of us 241 00:15:14,380 --> 00:15:19,375 think, a degraded experience over the past maybe five to 242 00:15:19,375 --> 00:15:23,695 seven years sharpening more as we approach the present. And 243 00:15:23,695 --> 00:15:27,295 when I say arms raised, I mean, it's the the old school pre AI 244 00:15:27,295 --> 00:15:31,950 content farms and the way that the poor recipe bloggers just 245 00:15:31,950 --> 00:15:34,750 started adding more and more stories about their great 246 00:15:34,750 --> 00:15:37,950 grandmothers to the beginnings of all their recipe posts to try 247 00:15:37,950 --> 00:15:41,870 to get the attention. There was a lot of really peculiar sort of 248 00:15:41,870 --> 00:15:47,085 perverse incentive minuet happening that made search 249 00:15:47,085 --> 00:15:49,405 harder and harder to deal with. If you were really good at 250 00:15:49,405 --> 00:15:53,005 search, you could still figure it out. But for, I think, casual 251 00:15:53,005 --> 00:15:56,925 use outside of any given person's area of expertise, it 252 00:15:56,925 --> 00:15:57,805 did get frustrating. 253 00:15:58,450 --> 00:16:01,330 But especially, Audrey, I appreciate you talking about the 254 00:16:01,330 --> 00:16:06,050 idea of taking action, moving everyone into increasingly 255 00:16:06,050 --> 00:16:10,930 enclosed knowledge spaces with prompts. It feels to me like 256 00:16:10,930 --> 00:16:16,185 we've rebuilt AOL, but it talks on one hand and that we're 257 00:16:16,185 --> 00:16:21,305 putting people into society wide wizards on the other, neither of 258 00:16:21,305 --> 00:16:25,545 which really constructs us as subjects with agency. So, 259 00:16:25,545 --> 00:16:29,220 obviously, I have, like, deep rooted philosophical problems 260 00:16:29,220 --> 00:16:33,380 with this. But I do think even though many of us might approach 261 00:16:33,380 --> 00:16:38,340 our refrigerators and say, I have two heads of wilted kale 262 00:16:38,340 --> 00:16:41,675 and three eggs. What am I having for dinner? 263 00:16:41,675 --> 00:16:46,555 There is a pretty strong sense among at least the people that 264 00:16:46,555 --> 00:16:50,635 that I know who are not in this nerd zone that that is 265 00:16:50,635 --> 00:16:54,075 qualitatively a different thing than what we do when we try to, 266 00:16:54,075 --> 00:16:57,470 for instance, educate our children or learn about the 267 00:16:57,470 --> 00:17:02,510 world or do research or do scholarship. So there's a pretty 268 00:17:02,510 --> 00:17:06,590 strong pushback in some aspects, but it bleeds into this. Yes, 269 00:17:06,590 --> 00:17:10,115 but there's the convenience factor with a class of things 270 00:17:10,115 --> 00:17:14,275 that perhaps can be answered by a machine and we're perfectly 271 00:17:14,275 --> 00:17:17,955 happy with that. Having those two modes, which in our own 272 00:17:17,955 --> 00:17:21,235 lives shift in and out based on how tired we are, what we're 273 00:17:21,235 --> 00:17:23,155 trying to accomplish, where we are and we're trying to 274 00:17:23,155 --> 00:17:27,860 accomplish it complicates this a lot and backed a lot of people 275 00:17:27,860 --> 00:17:30,820 into positions where they're doing things that they perhaps 276 00:17:30,820 --> 00:17:32,420 wish they were not doing. 277 00:17:33,060 --> 00:17:35,300 Audrey Evans: So those complications Erin brings up 278 00:17:35,300 --> 00:17:39,140 remind me of ideas I've heard you bring up before malls around 279 00:17:39,140 --> 00:17:42,555 ground truthing. Would you take us through some of that? 280 00:17:42,635 --> 00:17:45,275 Mols Sauter: One of the things I find interesting about LLMs at a 281 00:17:45,275 --> 00:17:48,235 theory level and sort of at a philosophy of technology level 282 00:17:48,315 --> 00:17:52,475 is the ways in which they counter expectations that we 283 00:17:52,475 --> 00:17:56,240 have about what computers do. And that makes the reaction to 284 00:17:56,240 --> 00:18:00,000 them complicated. And this sort of goes along with sort of the 285 00:18:00,000 --> 00:18:03,280 ground truthing and the ground truthing problem, which is a big 286 00:18:03,280 --> 00:18:07,280 problem in LLMs. Because if you think about what LLMs are and 287 00:18:07,280 --> 00:18:10,995 how they're constructed, their ground truth is not our truth, 288 00:18:10,995 --> 00:18:13,395 our ground truth. Our ground truth is based on the fact that 289 00:18:13,395 --> 00:18:14,515 we have bodies. 290 00:18:14,515 --> 00:18:18,595 They live in a physical environment. We have sensory 291 00:18:18,915 --> 00:18:22,950 access to that environment. But we expect a lot of things that 292 00:18:22,950 --> 00:18:26,150 we talk about in human communication, these bits of 293 00:18:26,230 --> 00:18:29,190 bits of sort of phonetic sound and text that we throw at each 294 00:18:29,190 --> 00:18:32,870 other, we expect those things to be related to that environment. 295 00:18:33,030 --> 00:18:35,505 That's our ground truth. We're like, this is these things have 296 00:18:35,505 --> 00:18:39,105 always coincided and when they don't coincide, that's bad. 297 00:18:39,105 --> 00:18:42,385 That's a lie. That's a falsehood. That's a mistake. 298 00:18:42,385 --> 00:18:47,425 It's not what we want. Computers have a very long history of 299 00:18:47,425 --> 00:18:51,370 being portrayed in sort of both broader culture, in politics, in 300 00:18:51,370 --> 00:18:52,730 science, and technology itself. 301 00:18:52,730 --> 00:18:57,850 It's like, this thing is tied to a set of ground truths that we 302 00:18:57,850 --> 00:19:00,810 call mathematics. And mathematics, it has a certain 303 00:19:00,810 --> 00:19:04,665 epistemic status. And we also expect it to progress through a 304 00:19:04,665 --> 00:19:07,545 series of logical steps. This is how it functions. We're going to 305 00:19:07,545 --> 00:19:08,265 walk through these steps. 306 00:19:08,265 --> 00:19:10,905 This is how a computer program functions. And the listeners 307 00:19:10,905 --> 00:19:14,025 cannot hear me put many scare quotes on many of those phrases. 308 00:19:14,105 --> 00:19:17,145 But that's sort of the general understanding of how computation 309 00:19:17,145 --> 00:19:25,150 works. But these systems do not exist as ground truths, we live 310 00:19:25,150 --> 00:19:28,830 in the outside world and we must refer to it in our speech, text, 311 00:19:28,830 --> 00:19:33,455 and action, or as pure mathematical systems of this is 312 00:19:33,455 --> 00:19:35,855 a logical progression through this set of mathematical steps 313 00:19:35,855 --> 00:19:41,135 and this lives in this epistemic frame. These systems exist in a 314 00:19:41,135 --> 00:19:45,160 big sea of language and linguistical associations, and 315 00:19:45,160 --> 00:19:49,160 that is an unground truthed situation. 316 00:19:49,640 --> 00:19:53,320 But internal to the LLM, the logic is not I need to produce 317 00:19:53,320 --> 00:19:55,560 things that are related to an external ground truth. The logic 318 00:19:55,560 --> 00:20:00,275 of the LLM is I need to present the most acceptable string of 319 00:20:00,275 --> 00:20:04,435 tokens based on strings of tokens that I have. The thing 320 00:20:04,435 --> 00:20:09,715 that is done is a performance. And it's a performance within 321 00:20:09,715 --> 00:20:14,230 the bounds of genre and how genre is constructed and 322 00:20:14,230 --> 00:20:19,670 reflected in these associative groupings of tokens. And this is 323 00:20:19,670 --> 00:20:23,110 a real problem for how we think about computers because we're 324 00:20:23,110 --> 00:20:25,765 used to coming to a computer and being like, math problem and 325 00:20:25,765 --> 00:20:27,765 having computer be like, math problem. 326 00:20:27,765 --> 00:20:30,565 Answer to math problem. Here you go. And I think this is why 327 00:20:30,565 --> 00:20:36,165 people reacted so aggressively or like pinned very strongly on 328 00:20:36,405 --> 00:20:39,660 it does math wrong Because that's the basic thing you 329 00:20:39,660 --> 00:20:42,460 expect a computer to do is you expect it to math right. The 330 00:20:42,460 --> 00:20:46,380 whole purpose of computers is math right. These do not math 331 00:20:46,380 --> 00:20:46,860 right. 332 00:20:46,860 --> 00:20:50,460 These other things write and they they other things write in 333 00:20:50,460 --> 00:20:53,725 a weird way. And I think that the weird way they do it is 334 00:20:53,725 --> 00:20:56,525 really relevant to, like, to how we think about those other 335 00:20:56,525 --> 00:21:00,685 things. But because they're in a computer and they present as a 336 00:21:00,685 --> 00:21:04,445 computer, it's causing, like, this huge collapse. 337 00:21:05,220 --> 00:21:08,340 Audrey Evans: So this reminds me of how this all started, Erin, 338 00:21:08,340 --> 00:21:12,420 with your posts on Blue Sky exploring how we orient 339 00:21:12,420 --> 00:21:16,740 ourselves and maybe even what postures might be most useful to 340 00:21:16,740 --> 00:21:20,020 take when we interact and engage with these systems. 341 00:21:20,465 --> 00:21:22,705 Erin Kissane: People talk about anthropomorphization. I don't 342 00:21:22,705 --> 00:21:25,345 think that's quite what's happening here. But when we see 343 00:21:25,345 --> 00:21:30,545 the computers stop mapping and we see them talking and 344 00:21:30,545 --> 00:21:34,545 especially when you said, Audrey, moving toward a back and 345 00:21:34,545 --> 00:21:40,250 forth for search, I think the back and forth is central to the 346 00:21:40,250 --> 00:21:45,290 weirdness of our moment in LLMs because we are experiencing 347 00:21:45,290 --> 00:21:49,050 them. Many of us, most of our population who touches this 348 00:21:49,050 --> 00:21:52,895 stuff is probably doing it in a conversational way and you only 349 00:21:52,895 --> 00:21:56,895 have conversations with entities. The entities are 350 00:21:56,895 --> 00:21:59,055 obviously not people. 351 00:21:59,455 --> 00:22:05,960 I don't want to get too bogged down in entity ness, but because 352 00:22:05,960 --> 00:22:09,960 of my particular background, which comes out of religion and 353 00:22:09,960 --> 00:22:14,440 folklore, this sure looks familiar to me. We actually have 354 00:22:14,440 --> 00:22:17,075 centuries of 355 00:22:17,395 --> 00:22:18,755 Mols Sauter: We have lots of this. 356 00:22:18,995 --> 00:22:23,475 Erin Kissane: Of cultural technology about how you deal 357 00:22:23,555 --> 00:22:29,715 with weird entities, and this predates psychology. We have a 358 00:22:29,715 --> 00:22:33,110 pretty strong tradition of things that are perhaps demons 359 00:22:33,110 --> 00:22:38,070 or that are perhaps gods or angels or spirits or creatures, 360 00:22:38,390 --> 00:22:42,470 things that provide us with divinatory knowledge. And in the 361 00:22:42,470 --> 00:22:45,030 same way that Miles was saying, you know, like, we expect 362 00:22:45,030 --> 00:22:49,595 computers to first be math, We kind of expect weird nonhuman 363 00:22:49,595 --> 00:22:55,115 entities that will talk back to us and give us advice to be a 364 00:22:55,115 --> 00:22:59,195 particular kind of thing. I think it is less like talking to 365 00:22:59,195 --> 00:23:03,070 a hallucinating entity and more like participating in 366 00:23:03,070 --> 00:23:07,710 hallucination. When you are in one of these weird zones, you 367 00:23:07,710 --> 00:23:09,790 know that you are not getting ground truth. 368 00:23:09,790 --> 00:23:12,110 You know you are someplace weird. And specifically, I think 369 00:23:12,110 --> 00:23:15,070 there's something that comes out of, like, cross cultural 370 00:23:15,230 --> 00:23:19,815 something sort of below fairy tales and into like spirit 371 00:23:19,815 --> 00:23:23,255 stories, folk stories, ghost stories, this stuff. The idea 372 00:23:23,255 --> 00:23:26,535 that the entities have a coherent system, but that you 373 00:23:26,535 --> 00:23:30,135 can't quite understand it. So your best bet is to sort out 374 00:23:30,135 --> 00:23:34,830 what kind of contract you can make with this class of entity 375 00:23:34,910 --> 00:23:38,110 or with animals or with other aspects of the natural world so 376 00:23:38,110 --> 00:23:40,670 that you have a crossing over point. None of these 377 00:23:40,670 --> 00:23:46,110 technologies allow for or permit just a continuous flow of back 378 00:23:46,110 --> 00:23:50,645 and forth with any of these entities or systems, I think 379 00:23:50,645 --> 00:23:55,765 that is instructive or it could be instructive for us about our 380 00:23:55,765 --> 00:24:00,740 own weaknesses to a continuous back and forth with something 381 00:24:00,740 --> 00:24:03,540 that's not human, that's not practicing according to our 382 00:24:03,540 --> 00:24:06,500 rules, that is not coming from embodied knowledge, that has 383 00:24:06,740 --> 00:24:09,540 either no ground truth or a ground truth that we can't 384 00:24:09,540 --> 00:24:10,500 really understand. 385 00:24:10,980 --> 00:24:13,700 Audrey Evans: I love this and I haven't heard anyone else 386 00:24:13,700 --> 00:24:17,755 articulate this quite like you do. I feel like you're inviting 387 00:24:17,755 --> 00:24:22,475 us into a more honest understanding of the 388 00:24:22,475 --> 00:24:27,835 relationality that's happening and inviting us to remember that 389 00:24:27,835 --> 00:24:31,650 the agency and where we define the relationship to this 390 00:24:31,650 --> 00:24:35,490 technology and how we interact with it has to start in our 391 00:24:35,490 --> 00:24:37,810 bodies and end in our bodies. 392 00:24:38,370 --> 00:24:41,330 Erin Kissane: And and I don't wanna take away from the broad 393 00:24:41,330 --> 00:24:46,765 political and anti labor aspect of this technology, but I do 394 00:24:46,765 --> 00:24:51,965 like it it frustrates me that the relational part, that 395 00:24:51,965 --> 00:24:55,050 conversation has been pretty crude. Yeah. And I think people 396 00:24:55,050 --> 00:24:58,650 are perhaps embarrassed to talk about it unless they somehow 397 00:24:58,650 --> 00:25:02,970 lack embarrassment and are happy to say, here are my 300 pages of 398 00:25:02,970 --> 00:25:06,250 chat log and why I think my Claudine is special. 399 00:25:06,810 --> 00:25:10,605 Mols Sauter: Just jumping off of the last point, the actual skill 400 00:25:10,605 --> 00:25:13,885 set you need first, I was like, you absolutely 100% need to have 401 00:25:13,885 --> 00:25:17,485 a very core understanding of the fae. This is this thing that is 402 00:25:17,645 --> 00:25:20,605 not telling me things in a way that I understand things to be 403 00:25:20,605 --> 00:25:23,100 telling me things. And you have to sort of be able to hold that 404 00:25:23,100 --> 00:25:25,100 in your head if you're gonna have an extended interaction 405 00:25:25,100 --> 00:25:28,220 with it. When I started looking at it and I started looking at 406 00:25:28,220 --> 00:25:31,340 it, it is like, what can it do? What is the thing that it is 407 00:25:31,340 --> 00:25:32,220 capable of doing? 408 00:25:33,515 --> 00:25:35,995 One of the first things that popped up to me was like, oh, 409 00:25:35,995 --> 00:25:39,275 what you need to make this work is discernment. I know the shape 410 00:25:39,275 --> 00:25:41,995 of what you should be making. You have made things in that 411 00:25:41,995 --> 00:25:44,715 shape, but something around this is is off. Something around this 412 00:25:44,715 --> 00:25:47,035 is not functioning. Something around this is not what I 413 00:25:47,035 --> 00:25:47,435 expected. 414 00:25:48,370 --> 00:25:53,570 Let's go figure out why this isn't this way. And I think most 415 00:25:53,570 --> 00:25:55,890 people use it like, people, especially with the way they're 416 00:25:55,890 --> 00:25:59,010 being distributed now, people engage with them in a one off. 417 00:25:59,010 --> 00:26:00,930 They're like, I have typed something into a search bar. I 418 00:26:00,930 --> 00:26:03,170 have gotten a result. I need to check something. 419 00:26:03,815 --> 00:26:05,895 Students will ask it to summarize things. One of the 420 00:26:05,895 --> 00:26:08,615 major projects we're working on this summer is about summary and 421 00:26:08,615 --> 00:26:11,495 how summary functions with that within different models. And 422 00:26:11,495 --> 00:26:14,935 also what a broader theory of summary is, which it turns out 423 00:26:14,935 --> 00:26:19,790 is quite difficult to pin down. I'm gonna bring in Lacan because 424 00:26:19,790 --> 00:26:23,950 I'm a theory sicko. Lacan has a concept called the subject 425 00:26:23,950 --> 00:26:25,070 supposed to know. 426 00:26:25,470 --> 00:26:29,390 And the subject supposed to know is the position occupied by the 427 00:26:29,390 --> 00:26:33,055 analyst in a psychoanalytic relationship. And the analysand 428 00:26:33,055 --> 00:26:37,055 enters into the analytical relationship and projects upon 429 00:26:37,055 --> 00:26:41,375 the analyst the position of the subject supposed to know. They 430 00:26:41,375 --> 00:26:44,655 are the person who so the the analysand is coming in and 431 00:26:44,655 --> 00:26:48,370 saying, I am full of weird shit. I am full of nonsense. I'm full 432 00:26:48,370 --> 00:26:49,250 of contradictory things. 433 00:26:49,250 --> 00:26:52,210 I'm full of compulsions. I'm full of, like, blockages. I'm 434 00:26:52,210 --> 00:26:57,090 full of all of this garbage, and I can't figure it out. That guy 435 00:26:57,090 --> 00:26:59,895 can. That's he's the subject supposed to know. 436 00:26:59,895 --> 00:27:03,095 He's gonna know. He he's gonna take all of my utterances and 437 00:27:03,095 --> 00:27:05,895 he's going to put them in order and he's gonna make it make 438 00:27:05,895 --> 00:27:09,575 sense to me. And the subject's most to know position is a 439 00:27:09,575 --> 00:27:13,440 collision of a bunch of things. It's a collision of what this 440 00:27:13,680 --> 00:27:16,800 what the analysis and previous experiences are with people, 441 00:27:16,800 --> 00:27:19,840 with people older than them, with therapists, with doctors, 442 00:27:19,840 --> 00:27:23,680 with teachers, whatever. Their knowledge about psychoanalysis, 443 00:27:24,000 --> 00:27:26,800 their knowledge their the environment that it is happening 444 00:27:26,800 --> 00:27:31,815 in, who the analyst presents as, how they've talked about 445 00:27:31,815 --> 00:27:32,615 themselves. 446 00:27:32,615 --> 00:27:35,255 All of this shit comes down into the subject supposed to know 447 00:27:35,255 --> 00:27:39,175 position. And importantly, the analyst doesn't know. That is 448 00:27:39,175 --> 00:27:41,735 their function is not to know you better than you know 449 00:27:41,735 --> 00:27:44,550 yourself. Their function is to sort of create a discursive 450 00:27:44,550 --> 00:27:47,990 construct where you can explain yourself to yourself and 451 00:27:47,990 --> 00:27:52,630 eventually do it in a way that makes sense to you. And I would 452 00:27:52,630 --> 00:28:00,065 like to posit that we approach computers in general, but 453 00:28:00,065 --> 00:28:04,065 specifically discursive computers and place them into 454 00:28:04,065 --> 00:28:05,905 the subject supposed to know position. 455 00:28:06,145 --> 00:28:08,785 I like this theory and it's very effective for me because it 456 00:28:08,785 --> 00:28:10,945 explains a bunch of different types of interactions. It 457 00:28:10,945 --> 00:28:14,110 explains the one off interactions. It explains the 458 00:28:14,110 --> 00:28:17,230 more the multi turn interactions. It explains why 459 00:28:17,230 --> 00:28:22,110 people think that LLMs can stand in for constituent data or all 460 00:28:22,110 --> 00:28:24,590 these other things because it is the subject supposed to know. 461 00:28:24,590 --> 00:28:27,675 And like the projection of the subject supposed to know, it is 462 00:28:27,675 --> 00:28:31,355 a it is a collapse of a bunch of stuff, including, like, the vast 463 00:28:31,355 --> 00:28:35,195 history of computing as computers, math, and math knows. 464 00:28:35,835 --> 00:28:40,715 And so when so when we enter into these interlocutor 465 00:28:40,715 --> 00:28:45,290 positions where what we are experiencing is a one to one 466 00:28:45,290 --> 00:28:50,410 interaction. But what is actually happening is a one, 467 00:28:50,410 --> 00:28:57,245 that's me, to one, the interface of the LLM, to large entity, 468 00:28:57,325 --> 00:28:59,885 meaning the training data and all of the training shit that's 469 00:28:59,885 --> 00:29:04,205 happened before to everyone. The guy you're talking to is the guy 470 00:29:04,205 --> 00:29:07,565 talking to everybody. Like, even though it seems like you it's 471 00:29:07,565 --> 00:29:09,645 not just talking to you, it's talking to everybody. Everyone's 472 00:29:09,645 --> 00:29:11,165 getting those weird weather metaphors. 473 00:29:11,440 --> 00:29:14,720 Everyone's getting delve. Everyone's getting these words 474 00:29:14,720 --> 00:29:18,720 over and over again. It occupies these like, it's stepping into 475 00:29:18,720 --> 00:29:24,255 these existing social and cultural constructs and 476 00:29:24,255 --> 00:29:28,415 animating them in a way that is deeply destabilizing. So I think 477 00:29:28,415 --> 00:29:31,055 Erin is totally right that we have all of these stories. We 478 00:29:31,055 --> 00:29:33,775 have all these ways of understanding these things. 479 00:29:34,015 --> 00:29:36,895 Erin Kissane: Thinking about subjects supposed to know in 480 00:29:36,895 --> 00:29:43,180 this sort of post therapeutic turn of society, again, I get 481 00:29:43,180 --> 00:29:49,900 pulled back into who we asked to do that kind of organizing work 482 00:29:49,900 --> 00:29:56,375 for us before that turn, were frequently whatever kind of 483 00:29:56,375 --> 00:30:03,095 spiritual advisors we depended on. I think that work that Malz 484 00:30:03,095 --> 00:30:09,800 is doing to break down our own positioning, our expectations of 485 00:30:09,800 --> 00:30:14,120 the role, really important and could be really useful in 486 00:30:14,120 --> 00:30:19,165 helping people pull out of the mode that you might use if you 487 00:30:19,165 --> 00:30:23,245 bring these kinds of problems to someone who is either an 488 00:30:23,245 --> 00:30:27,245 explicitly spiritual authority. I am bringing it to my priest. I 489 00:30:27,245 --> 00:30:30,605 am bringing it to my minister. I'm bringing it to someone in my 490 00:30:30,605 --> 00:30:35,180 zone who has ethical authority if they're not religious, to an 491 00:30:35,180 --> 00:30:35,900 elder. 492 00:30:36,300 --> 00:30:39,980 Those people are intended to give you different kinds of 493 00:30:39,980 --> 00:30:43,740 answers. Those people are intended to give you the 494 00:30:43,740 --> 00:30:49,675 position of cultural wisdom and authority and advice that will 495 00:30:49,675 --> 00:30:55,195 help you find your way out of whatever quandary you're in. 496 00:30:56,155 --> 00:31:00,090 That's a really dangerous posture to bring to these 497 00:31:00,090 --> 00:31:05,530 machine systems. So I think that kind of picking apart is 498 00:31:06,090 --> 00:31:09,050 essential work and I'm excited that it's happening. I also 499 00:31:09,050 --> 00:31:12,890 snagged a little bit on discernment because it's such a 500 00:31:13,125 --> 00:31:15,125 it's it's both such a great concept. 501 00:31:15,125 --> 00:31:18,245 And I don't know if this will be clear to those of us who didn't 502 00:31:18,245 --> 00:31:25,205 grow up as church kids, but the age of discernment is a big deal 503 00:31:25,720 --> 00:31:29,720 in at least the forms of Protestant Christianity that I 504 00:31:29,720 --> 00:31:34,120 grew up in. I think there are very similar structures across 505 00:31:34,120 --> 00:31:38,520 at least a bunch of the Christian tradition. But there's 506 00:31:38,855 --> 00:31:44,055 the idea that until children hit the age of discernment, you need 507 00:31:44,055 --> 00:31:47,095 to treat them as a slightly different kind of creature. They 508 00:31:47,095 --> 00:31:51,815 don't have moral reasoning. They don't have the lived experience 509 00:31:53,040 --> 00:31:56,640 to confront really difficult questions. 510 00:31:57,920 --> 00:32:03,680 Where we reconstruct discernment for this moment, what 511 00:32:03,680 --> 00:32:08,905 discernment is going to be for the children, teenagers, 512 00:32:08,985 --> 00:32:14,505 students, all of us approaching these weird systems feels like a 513 00:32:14,505 --> 00:32:16,185 really live question to me. 514 00:32:16,185 --> 00:32:18,665 Mols Sauter: I think can I can I just give a brief defense of, 515 00:32:18,665 --> 00:32:20,810 like, a thing that I say a lot people that frequently yell at 516 00:32:20,810 --> 00:32:23,130 me for, but I'm just like, no, this is a very this is how I 517 00:32:23,130 --> 00:32:26,410 think about it and I refuse to stop? Yeah. Okay. So Susan 518 00:32:26,410 --> 00:32:29,530 Sontag in On Photography. At the end of On Photography, she's 519 00:32:29,530 --> 00:32:32,010 presenter at the end of the essay that that is the the money 520 00:32:32,010 --> 00:32:33,290 essay in On Photography. 521 00:32:33,370 --> 00:32:38,045 She says, okay. There are now both there's the ecology of 522 00:32:38,045 --> 00:32:41,645 reality and the ecology of images. And over the entire part 523 00:32:41,725 --> 00:32:44,925 over this entire essay, she has argued, the ecology of images is 524 00:32:44,925 --> 00:32:49,410 basically taking over and re and, like, re accounting for the 525 00:32:49,410 --> 00:32:52,530 ecology of reality. And this is a problem. She does not like 526 00:32:52,530 --> 00:32:53,010 this. 527 00:32:53,490 --> 00:32:56,690 And I think on photography is one of the most important essays 528 00:32:56,690 --> 00:33:00,930 you can read about data and data systems, not just computer 529 00:33:00,930 --> 00:33:04,975 vision systems, but any data systems. Instead of having an 530 00:33:04,975 --> 00:33:09,695 ecology that I can access and look at and contest, you have 531 00:33:09,855 --> 00:33:13,615 created a self referential system that only relies on 532 00:33:13,615 --> 00:33:16,700 itself to define its own reality. And that is what 533 00:33:16,700 --> 00:33:20,220 psychosis is. That is when you have a self referential reality 534 00:33:20,220 --> 00:33:24,140 that is not available for intervention. When people 535 00:33:24,140 --> 00:33:28,540 attempt to intervene, that input is taken and translated into the 536 00:33:28,540 --> 00:33:32,095 self referential system and is not permitted to destabilize it. 537 00:33:32,575 --> 00:33:36,815 And that's what this is. That's these are like, when you have 538 00:33:36,815 --> 00:33:40,655 nested realities in this way, that is what is occurring. So 539 00:33:40,655 --> 00:33:43,295 when I sit when we're talking about the hallucination framing, 540 00:33:43,730 --> 00:33:45,810 the reason why I don't necessarily like the 541 00:33:45,810 --> 00:33:50,290 hallucination framing is because, a, of all things become 542 00:33:50,290 --> 00:33:52,690 hallucinations only when they are offered to someone who is 543 00:33:52,690 --> 00:33:56,770 outside of the self contained system. So it could be that yes. 544 00:33:56,770 --> 00:33:58,930 You're receiving something that you are defining as a 545 00:33:58,930 --> 00:33:59,410 hallucination. 546 00:33:59,785 --> 00:34:02,585 Internal to the system though, that is that is the reality. 547 00:34:02,585 --> 00:34:05,225 That is the that is the natural production of the context that 548 00:34:05,225 --> 00:34:10,425 it has. I think, like, when we're talking about systems and 549 00:34:10,425 --> 00:34:13,465 interactions and ways of interacting with things that are 550 00:34:13,465 --> 00:34:18,010 fundamentally occupying separate and contained realities from the 551 00:34:18,010 --> 00:34:23,690 one that the we all exist in, you really can't discount that. 552 00:34:24,250 --> 00:34:27,370 Erin Kissane: I think my intervention on top of that is 553 00:34:27,370 --> 00:34:33,605 to loop back around to this question the Dodri brought us at 554 00:34:33,605 --> 00:34:38,885 the very beginning about what happens when search stops being 555 00:34:38,885 --> 00:34:44,560 a list of references and starts being a conversation in a closed 556 00:34:44,560 --> 00:34:49,840 and as you just emphasized, actually quite sharply bounded 557 00:34:49,840 --> 00:34:54,880 subset of material. And of course, what's in the the 558 00:34:54,880 --> 00:34:57,935 training data is not everything that's on the Internet. It is a 559 00:34:57,935 --> 00:35:00,895 lot of awful things that are on the Internet, but even that is 560 00:35:00,895 --> 00:35:03,935 not everything and what's on the Internet is a subset of 561 00:35:03,935 --> 00:35:06,815 everything. So there's all that shaping that was already going 562 00:35:06,815 --> 00:35:11,230 in. And then we have these and I actually because I'm looking at 563 00:35:11,230 --> 00:35:14,350 the technical definitions, I was looking into psychosis and 564 00:35:14,350 --> 00:35:17,070 hallucinations next to each other because of course in human 565 00:35:17,070 --> 00:35:19,070 psychology they're often related. 566 00:35:19,870 --> 00:35:26,125 We are running these systems that work in this closed way 567 00:35:26,525 --> 00:35:31,485 over this highly shaped dataset. Additionally, there are the 568 00:35:31,485 --> 00:35:36,685 incentives that the corporations producing these systems are 569 00:35:36,685 --> 00:35:41,950 applying. It becomes impossible for those of us on the search 570 00:35:41,950 --> 00:35:45,870 box end to understand which of these factors we're 571 00:35:45,870 --> 00:35:49,230 encountering, which mixture of these factors we encounter in 572 00:35:49,230 --> 00:35:53,805 any given interaction, which is also profoundly destabilizing 573 00:35:53,805 --> 00:35:59,405 and leads, I think, to a lot of folksonomy of knowledge. This is 574 00:35:59,405 --> 00:36:03,325 happening because someone wanted it to happen versus this is 575 00:36:03,325 --> 00:36:09,510 happening because the machines have a psychotic construct of 576 00:36:09,510 --> 00:36:12,550 moving through the world. This one is happening because 577 00:36:12,550 --> 00:36:14,390 training data is what it is. 578 00:36:14,710 --> 00:36:18,070 It's it's impossible for us to know. Therefore, we are going to 579 00:36:18,070 --> 00:36:22,605 make so much shit up about what is happening. At the same time, 580 00:36:22,605 --> 00:36:27,005 we are also being, I think, substantially destabilized in 581 00:36:27,005 --> 00:36:31,005 our understanding of our own ability to think and know 582 00:36:31,005 --> 00:36:34,605 things. That's a pretty volatile combination when you are feeling 583 00:36:34,605 --> 00:36:41,120 destabilized as someone who may have less agency or subjectivity 584 00:36:41,120 --> 00:36:45,040 than you would expect and you're encountering all this stuff. It 585 00:36:45,040 --> 00:36:47,520 feels like a powder keg to me culturally. 586 00:36:48,175 --> 00:36:52,975 I don't think we tend to see great things happen in eras 587 00:36:53,375 --> 00:36:58,895 characterized by really extensive spiritual malaise 588 00:36:58,895 --> 00:37:04,260 about our ability to know things and take action. Not to end on a 589 00:37:04,260 --> 00:37:07,220 bummer, but I do think these things are worth watching 590 00:37:07,220 --> 00:37:12,260 closely as we bring all of this to everyone whoever touches 591 00:37:12,580 --> 00:37:14,260 search or asks their phone a question. 592 00:37:14,675 --> 00:37:17,235 Mols Sauter: Just a shout out quick shout out to Nick Siever 593 00:37:17,235 --> 00:37:19,875 and the book and his book Computing Taste and the concept 594 00:37:19,875 --> 00:37:23,235 of the Heterogloss Corporation. That is a really good sort of 595 00:37:23,235 --> 00:37:26,515 set of ideas to think through like how a bunch of different 596 00:37:26,515 --> 00:37:29,555 things get combined, like Yeah. Collapsed down into the one 597 00:37:29,860 --> 00:37:33,700 result, the one interface, the one he's talking about a company 598 00:37:33,700 --> 00:37:38,660 that is definitely maybe not Spotify, but great So good. Also 599 00:37:38,660 --> 00:37:39,380 super readable. 600 00:37:43,995 --> 00:37:48,235 Audrey Evans: This was my dream interview. I had so much fun. 601 00:37:48,555 --> 00:37:49,995 Mols Sauter: It's the interview with the reading list. 602 00:37:49,995 --> 00:37:52,155 Audrey Evans: Yeah. Thank you both so much. 603 00:37:52,155 --> 00:37:53,355 Erin Kissane: Thank you, Audrey. 604 00:37:55,595 --> 00:38:01,080 Audrey Evans: You can find Mauls and Aaron all over the web, but 605 00:38:01,080 --> 00:38:05,240 I'm gonna lead you to where this all started on Blue Sky when I 606 00:38:05,240 --> 00:38:08,760 stumbled across a really interesting conversation they 607 00:38:08,760 --> 00:38:14,195 were having. You can find Mauls at oddletters dot bsky dot 608 00:38:14,195 --> 00:38:21,715 social and Erin at kisane.my@proto.social. 609 00:38:38,835 --> 00:38:53,290 Mike Rugnetta: That is the news we have for you this week. We're 610 00:38:53,290 --> 00:38:57,050 gonna be back here in the main feed on or around Thursday, July 611 00:38:57,050 --> 00:39:03,975 2. How do we love $4 a month for a Never Post membership? Let me 612 00:39:03,975 --> 00:39:14,055 count the ways. One, two, three, four. 613 00:39:16,055 --> 00:39:20,280 That's it, I guess. Neverpo dot s t. Become a member today. 614 00:39:21,640 --> 00:39:24,600 Never Post's producers are Audrey Evans, Georgia Hampton, 615 00:39:24,600 --> 00:39:28,040 and the mysterious doctor, first name, last name. Our senior 616 00:39:28,040 --> 00:39:29,400 producer is Hans Buto. 617 00:39:29,400 --> 00:39:32,845 Our executive producer is Jason Oberholzer. Show's host, that's 618 00:39:32,845 --> 00:39:39,565 me, is Mike Rugnetta. Moving variant ornithography of those 619 00:39:39,565 --> 00:39:44,525 uninitiated made into memory by the me briefly incarnate. Full 620 00:39:44,525 --> 00:39:48,540 of myself on successive nights dense and alone sings you back. 621 00:39:49,020 --> 00:39:53,420 Need keeps the book of dying open, the language common after 622 00:39:53,420 --> 00:39:53,980 all. 623 00:39:54,620 --> 00:39:59,925 Excerpt of fourth fourth from the whole note by Gil Ott. Never 624 00:39:59,925 --> 00:40:03,205 Post is a production of charts and leisure and it's distributed 625 00:40:03,205 --> 00:40:05,685 by Radiotopia.