Since I'm like public journaling now instead of just allowing thoughts to pass through my head without any reinforcement and then showing up to hubski like "oh, I don't have anything", I'll give an example. "If I tried to LLM at work". There's a global model of the magnetosphere and surrounding solar wind environment that I run through a public website. I query the model for a certain day or time that I want (step 1). Wait a few days, then I look through the results and do the science (step 2). For step 1, there is no benefit in having a program input the date and time with a few choices that I make for which sub-components of the magnetosphere model I want to use, because it takes about five minutes. For step 2, the way that I look through the data requires an entire methodology in which I'm using outputs from the model to re-input back into the next time-step for visualization. I'm tracing magnetic field lines through time/space and the magnetosphere as it convects (I've automated it using a python webcrawler and maths to produce a movie). The idea that I could simply ask an LLM to do this is pretty funny. It's so specialized that I can guarantee it would fail immensely to know wtf I meant when I said "take the results from this model run and show me a movie of magnetospheric convection. I want bundles of magnetic field lines that pass through the reconnection site near satellite XYZ emphasized". I think the amount of additional information I would need to feed it for the thing to even come close is infinite, because it's probably never going to give me something good. More on that below. But let's say that it does. It's the game of "how do I know it's right?" again. I've gotta inspect all of the code that it wrote to do it, and I can guarantee that it's gonna be an implementation that's a way different structure than mine. I'm going to put in so much effort checking it that I'm not going to save an iota of time. OK, so I have my video, one way or the other. I can now look through it and do the actual science, linking it into an analysis of data from that satellite. There is simply no fucking way that any LLM or AGI on the foreseeable horizon could do this. Doing the science means comparing the new m'sphere model outputs to the existing data analysis, linking new interesting/publishable physics of the two, discussing how this is different or similar to previous studies, and thinking about how the results can be applied towards the next step. It requires a deep understanding of how this contributes to the field. This is at least approaching ASI territory. Furthermore, for the science, the LLM or whatever it is has no interest in images. It only cares only about model outputs. It would actually have to perform the conjugate of what I have to, and take the images from previous movies of magnetosphere convection and put them into a form for comparison with the magnetosphere model output data. The whatever it is will have to know how to transform the data into formats suitable for comparison, and then it'll have to have correctly ingested the publishing record to form a pseudo-understanding of everything. Can't imagine the lengths it would have to go to output something like "we can see that if the only difference is a Y-component reversal of the upstream magnetic field in the solar wind, the reconnection site moves southward towards the spacecraft, because the X-line is shifting to accommodate cusp reconnection relocating from the cusps on the dawn and north and dusk and south quadrants to the dawn, south and dusk, north quadrants, respectively". Would the Whatever know that it'd be good to run the magnetosphere model I used for the period of time used in the previous study, which used a completely separate m'sphere model, to factor in the differences between the two models that might explain the behavior instead? Does it know that it's important to comment on the distance from the satellite to the reconnection site? Is the data analysis conclusion that the satellite is at a reconnection site actually wrong? Are there shortcomings in the m'sphere model that help explain why the m'sphere model's reconnection site differs from where we actually found it? It's obviously not advisable to expect this inside of two or several decades. Maybe it could build me a movie, but I doubt it. Unless I am guaranteed a running instance of my efforts to coach it is preserved and always available should I achieve a successful/correct movie once, or that any new pseudo-understanding I had to lead it to is properly assimilated into the root system, there's no reason to even begin trying. Correct me if I'm wrong, but that's not something publicly available yet, and I can see massive hurdles to it ever happening. lol, what am I gonna say? "That's right! You finally did it. Now, don't forget how to do this the next time I ask, I don't want to have to spend another seven months filling in the gaps in your understanding of this again"? Hahahha "Filling in gaps of understanding" deserves a dissection, because it's more general, not just for physics or science, but for anything. The process looks like hell. Because, like we've said, the LLM doesn't know what's "correct", it's not going to ask you any substantive questions. It's going to output what it outputs, and you'll have to look at the outputs, and tell it why it's wrong. Iteratively. Having it fix one thing could break another. It could even infinitely diverge instead of ever converging on the solution you want it to. This all assumes that you know what you're looking for, what "right" means. And then, even if it does get things right, yeah, unless you work at the company that owns the LLM, it's all forgotten when you close the instance. Job security. Job security for all!
I had a discussion with an old buddy about LLMs yesterday. He's writing fiction and is using ChatGPT like a rented mule. He's got a character who's modeled on Andrew Tate but he wants him to be annoying, not a villain, so he'll type "give me ten things a sexist asshole would say about women that aren't awful." He's got a character who's a vampire so he'll type "give me a list of insults a vampire would use against townsfolk." Or he'll be analyzing plot points and he'll say "give me a list of movie scenes that would radically change the movie if they were absent." In each one he goes through and picks what he likes. In the last one he argues with it. I pointed out that he's basically using ChatGPT like an extended thesaurus and he agreed. I also pointed out that if you ask an LLM "give me the stochastic mean of this vector through a set of points" you are using the LLM as it was intended to be used - it will give you the mediocrity every time and, because it's basically a hyperadvanced Magic 8 Ball every now and then it will be brilliant. But - I pointed out - when you ask it for an opinion it will fall down every time because it has absolutely no handles on any of its inputs and outputs. You can't ask it to tell you what scenes are crucial because it has no understanding of any of the concepts underneath. What it has is a diet of forum posts that it will never give you straight. Shall we play "how can chatGPT do my job?" 'cuz they've been trying to AI automate my job forever. See this guy? they were about $1500 back in '94. And what they do is analyze the audio signal passing through them looking for feedback, and then they drop one of eight filters on it. You can adjust the sensitivity to feedback, you can adjust the latch, you can adjust the release, you can adjust the aggressiveness. They were really big until about 2005 or so when it became cheap and easy to TEF sweep a room and ring it out to EQ out the frequencies that cause things to ring - I'm sitting here surrounded by ten speakers at 85dB and having spent an afternoon mapping and collating and inserting between 4 and 15 filters each channel I can't get feedback if I hold a condenser in front of left main. Could an AI have done that? fuck yeah. That would have been delightful. But not without me moving the mic sixty times so what time am I actually saving? That active seeking feedback reduciton thing has made it into machine tools - each servopak on my mill has more filters than that Sabine. And in general, the approach everyone takes is "set as many as you need to kill steady-state, use the roaming ones carefully" because who knows what modes you'll run into with this or that chunk of aluminum strapped down getting chewed up. Everything I've got is already a waveform. We've been using Fourier transforms to operate on them for 40 years. My life is nothing but math. And despite the fact that GraceNote has literally released every song they know about as training data, telling the AI "make my mix sound better" still fucking failwhales. Like, on a basic, simple level. It understands what the sonogram of a song should sound like but that's like reconstructing a fetus from an ultrasound. What you get is uncanny valley nightmare fuel. I don't need the mediocre middle of a million mixes, I need excellence. And excellence comes from humans because it is, by definition, not the mean. Anyone expecting that a machine purpose-built to give you a statistical average can give you only the good outliers is going to be disappointed for the simple fact that the machine doesn't understand "good" or "bad" it understands "highly rated" or "much engaged with." The machine thinks this is the best Jurassic Park cover ever made: And the only way you can deal with that is to nerf it out on a case-by-case basis. You could argue that LLMs are good for facts but not opinions but the problem is its method for handling facts only works for opinions. Are they useful? Yes. Are they a tool that will make big changes to a few industries? I don't see how they can't. Am I honestly excited to see their actual utility? You damn betcha. But where the world is now is this: People who don't understand AI inflicting it on people who don't need AI to the detriment of people who don't want AI. That's it. That's the game.
Ahh, of course, the feedback thing. I don't do anything live, so I can just get away with a pretty simple gate and headphones. No chance of loops. Hadn't really thought about how I would suppress feedback loops without killing the channel or at least lowering the volume. But now I completely get it. I got really close to connecting the dots a long time ago when I suggested basically TEF in a convo with you a few years back. My mistake was thinking about mixing. I was thinking about minimizing phase cancellations as a function of frequencies. But duh: My co-worker would bolt a plasma spectrometer with accelerometers on it to a vibration table with some special isolators between the instrument and mounting baseplate, and we'd shake them with a sine sweep survey starting from like 1 Hz up through, I dunno, 40 kHz or something like that, and a power spectrogram level was input to govern the amplitude around each frequency. JUST like what you're doing with mics? We do it too. We'd already calculated the approximate normal modes of the instrument from 3D CAD models (we used Ansys), and so we notched the input frequency spectral energy around the normal modes so we don't overdrive the thing during vibe testing. And then we shake it with the launch environment, a white-noise spectrum, still modestly notched around the normal mode frequencies (which might have needed slight readjustments from the sine sweep results). By the way, at GSFC, they have like a 10 foot diameter gramophone to just blast shit with. I'd guess it was for Saturn V's, hahah, but I don't know! Didn't get the story. (edit: ohhhhh, I think it might've been for cleaning, especially considering that it was being kept in one of the anterooms bordering a clean room. They must be using the thing to knock any loose particles off of equipment or instruments with sound. We did the same thing with an ultrasonic bath after de-greasing parts with trychloride, before the final isopropyl wipe down. They'd soundblast it after that. Probably a pretty clean room.) Which has its uses, heh, though perhaps mostly uncommercializable. Absolutely agree. The LLM is navigating topological features inside a parameter space. With boundaries, and curvature, yeah. It's what I'm doing for the magnetosphere, actually. Same kind of idea. Except with I dunno maybe a billion axes instead of the four I use. But yeah, sometimes if you move just a little bit in the parameter space from where you started last time, or you start off in a slightly different direction, the topology might map to some drastically different places. Occasionally they will conjoin into beauty. AISI; artificial idiot savant intelligence. Hadn't heard any AI tunes yet, and figured there was good reason for it. I don't go looking for them, and a really good one would have found its way to me by now if it existed. We don't, agreed. I only want it for selfish reasons. And I only want it if I can feel assured it isn't going to cripple society. So I don't want it. Nvm. Feels like we're all getting a better handle on the level of complexity to expect though. It'll change. Hopefully not too fast, this has apparently been jarring enough for the world already, but AGI in two years? I just don't think so, and I'm 100% sure that ASI isn't only three years out.What you get is uncanny valley nightmare fuel
I also pointed out that if you ask an LLM "give me the stochastic mean of this vector through a set of points" you are using the LLM as it was intended to be used - it will give you the mediocrity every time and, because it's basically a hyperadvanced Magic 8 Ball every now and then it will be brilliant.
...people who don't need AI...
that sounds so fucking awesome Well what you're doing is ringing out the frequency response, right? You're trying to find constructive modes that are going to fuck you over while strapped in a rocket. You do that with an equalizer if it's sound or filters if it's an electromechanical system. I've linked this before, the eldritch magic starts at 3:35: For the record the last time I used ANSYS it was a command-line program that ran on a DEC Alpha. that sounds so fucking awesome You are grossly underestimating the ease with which bad mixes can be produced. The computer music cats have been doing "generative music" for a long time. It's easy as shit and doesn't require an LLM. Most of them are some form of neural network somewhere; "random ambient generator" has been an off-the-shelf product category for 20 years. Here's a free plugin for Kontakt. Here's a walk-through for Ableton.My co-worker would bolt a plasma spectrometer with accelerometers on it to a vibration table with some special isolators between the instrument and mounting baseplate,
and we'd shake them with a sine sweep survey starting from like 1 Hz up through, I dunno, 40 kHz or something like that, and a power spectrogram level was input to govern the amplitude around each frequency. JUST like what you're doing with mics? We do it too.
We'd already calculated the approximate normal modes of the instrument from 3D CAD models (we used Ansys)
By the way, at GSFC, they have like a 10 foot diameter gramophone to just blast shit with.
Which has its uses, heh, though perhaps mostly uncommercializable.
Hadn't heard any AI tunes yet, and figured there was good reason for it. I don't go looking for them, and a really good one would have found its way to me by now if it existed.
Absolutely. The normal modes. As it goes, first is the worst, second is the best, third is the one with the treasure chest. Sometimes it's "hairy chest", depends on the elementary school. When people use generative stuff in music well, it's noted. One of the most ridiculous arpeggio parts ever was made with Omnisphere's arpeggiator and then meticulously adapted for guitar. Probably took a little bit of practice (the rest of my life, in my case).Well what you're doing is ringing out the frequency response, right?
Dunno, probably not, but I think you could instantiate one that can when they can and freeze its learned ability, so the whole hoping it doesn't forget might go away. But I have no idea. Don't write that much code or work with raw data these days, so bibliographic aid is just about all it can do for me in an hour of need. Otherwise, it's about as tangential to my goings-on as it can get. When I tried that 'explain paper' site, it left enough of a distaste for me to roll eyes and move past. Between absolutely fucking insisting that some unrelated mathematical concept[0] is absolutely crucial to explain my question and rephrasing a circular argument until I got bored and left, I probably won't bother again for quite a while. Unfortunately, the above experience mean I'm unlikely to trust LLMs with stuff I don't know a lot about. Also, I kinda regret writing anything in this thread and will probably just add more tags to my ignored list. Fun company notwithstanding - too much hassle, too few fucks left. [0] - I wrote and deleted 900 word footnote of jargon about orbits of the coadjoint representation groups and operators in de Sitter space, so let's pretend I said Tits index and wiggled my eyebrows in an amusing way.Correct me if I'm wrong
That is the only way to fly, in my opinion, and we haven't discussed this much (edit: well nah we kinda have), but people aren't going to use it like that, obviously. Don't blame you for any filterings. I kinda like livening up this place. It's LLM season on hubski, baby. But one last quick story! I'm a couple miles from home standing in line to order a burger (probably in flip flops again) and a guy gets in the to-go line. Says "Order for so-and-so", and the cashier checks the order tickets. Nothin'. He says "I called such and such number". She refers to some post-its behind her, and sees that it's the other branch across town that he called and ordered from. He then says "watch", pulls up his phone, and goes "Siri, call Restaurant X on Street Y" (where we are), and it was replicable, it dialed the other branch again. He goes "so it's not my fault. I should get some food for free, I already paid". And I think he did. And he cut everyone in line. I wasn't in a hurry, it was nice to have front row seats for such a prescient demonstration. It's gonna be a fun time.... I'm unlikely to trust LLMs with stuff I don't know a lot about.
When every foodhole in Warsaw connected with delivery service overnight, outgoing orders had much much higher priority. So, during pandemic, you had a crowd of deliverers, normal line that moved at snail's pace, and a nearby crowd of people who placed their orders in an app to game the system. This lead to a situation where people from the last group placed order to <restaurant's address> and added comments like "I'm the one wearing a brown hat with a gigantic pompom" or "I'm already behind you." Insert something about follies of idiots with access technology. I don't know, I barely slept since Friday.I wasn't in a hurry, it was nice to have front row seats for such a prescient demonstration.
Yeah. Gonna be a lot of LLM Florida man stories. Same. But I do like checking back in here when I hit a roadblock at work. It's synergistic. Good luck with your coming week. Mine's gonna be crunch time, but I think I'm almost ready. Peaceeeeebarely slept since Friday