I think this needs to be repeated, since I tend to be quite negative about all of the 'AI' hype:
I am not opposed to machine learning. I used machine learning in my PhD and it was great. I built a system for predicting the next elements you'd want to fetch from disk or a remote server that didn't require knowledge of the algorithm that you were using for traversal and would learn patterns. This performed as well as a prefetcher that did have detailed knowledge of the algorithm that defined the access path. Modern branch predictors use neural networks. Machine learning is amazing if:
The problem is too hard to write a rule-based system for or the requirements change sufficiently quickly that it isn't worth writing such a thing and,
The value of a correct answer is much higher than the cost of an incorrect answer.
The second of these is really important. Most machine-learning systems will have errors (the exceptions are those where ML is really used for compression[1]). For prefetching, branch prediction, and so on, the cost of a wrong answer is very low, you just do a small amount of wasted work, but the benefit of a correct answer is huge: you don't sit idle for a long period. These are basically perfect use cases.
Similarly, face detection in a camera is great. If you can find faces and adjust the focal depth automatically to keep them in focus, you improve photos, and if you do it wrong then the person can tap on the bit of the photo they want to be in focus to adjust it, so even if you're right only 50% of the time, you're better than the baseline of right 0% of the time.
In some cases, you can bias the results. Maybe a false positive is very bad, but a false negative is fine. Spam filters (which have used machine learning for decades) fit here. Marking a real message as spam can be problematic because the recipient may miss something important, letting the occasional spam message through wastes a few seconds. Blocking a hundred spam messages a day is a huge productivity win. You can tune the probabilities to hit this kind of threshold. And you can't easily write a rule-based algorithm for spotting spam because spammers will adapt their behaviour.
Translating a menu is probably fine, the worst that can happen is that you get to eat something unexpected. Unless you have a specific food allergy, in which case you might die from a translation error.
And that's where I start to get really annoyed by a lot of the LLM hype. It's pushing machine-learning approaches into places where there are significant harms for sometimes giving the wrong answer. And it's doing so while trying to outsource the liability to the customers who are using these machines in ways in which they are advertised as working. It's great for translation! Unless a mistranslated word could kill a business deal or start a war. It's great for summarisation! Unless missing a key point could cost you a load of money. It's great for writing code! Unless a security vulnerability would cost you lost revenue or a copyright infringement lawsuit from having accidentally put something from the training set directly in your codebase in contravention of its license would kill your business. And so on. Lots of risks that are outsourced and liabilities that are passed directly to the user.
And that's ignoring all of the societal harms.
[1] My favourite of these is actually very old. The hyphenation algorithm in TeX trains short Markov chains on a corpus of words with ground truth for correct hyphenation. The result is a Markov chain that is correct on most words in the corpus and is much smaller than the corpus. The next step uses it to predict the correct breaking points in all of the words in the corpus and records the outliers. This gives you a generic algorithm that works across a load of languages and is guaranteed to be correct for all words in the training corpus and is mostly correct for others. English and American have completely different hyphenation rules for mostly the same set of words, and both end up with around 70 outliers that need to be in the special-case list in this approach. Writing a rule-based system for American is moderately easy, but for English is very hard. American breaks on syllable boundaries, which are fairly well defined, but English breaks on root words and some of those depend on which language we stole the word from.
@david_chisnall The pattern-matching is making Google searches more useful for me.
But I am entirely against referring to any of this stuff as ‘artificial intelligence’. It actually is not even an ATTEMPT to solve the problem of artificial intelligence. It is only mistaken for an attempt to solve that problem.
I have a simple proof of this: artificial intelligence cannot possibly be reached by ‘language models’ of any kind. Why not? Because a human is nearly the same as a language-less ape.
@david_chisnall Everything that people mistake for ‘AI hallucinations’ and so forth are easily understood if you simply view all the ‘AI’ not as ‘AI’ but as pattern processing algorithms.
True artificial intelligence would be an entirely different problem. The people doing the pattern processing are probably incapable of understanding the difference between the two problems, however.
@david_chisnall Apes truly are language-less BTW. People sometimes object. But true language has infinitely recursive structure. No animal has language in this sense.
And human intelligence MUST be a variant of ape intelligence. Therefore must have language only as an adjunct facility, not as a foundational one.
I expect LLMs never to be any good at mathematics. They claim otherwise, but what they demonstrate is only that it can solve high school math tests, which isn’t the same thing.
@david_chisnall Being good at math requires proprioceptive intelligence.
@chemoelectric I would like to hear more about this connection between proprioception and math. (I have terrible proprioception, but math is easier than language for me.)
@hosford42 It could also be ‘visualization’.
Most mathematics is really geometry. Even non-geometric stuff is usually solved by converting it to geometry. Whathisface proved Fermat’s last theorem by geometry.
@chemoelectric That makes more sense to me. I'm a *very* visual thinker, and it has aided my mathematical intuition enormously.
@hosford42 @chemoelectric
I'm not good at maths, I'm not good at thinking visual things like: a dice spread out as a plane. But when someone talks in metaphors I see the images in my head. 😅 I didn't know proprioception and math were linked. I would like to hear more about this!
I mean proprioception in the sense of navigation. Knowing left and right. That sort of thing. Which obviously fully healthy apes are very, very good at.
(I’m not so extremely healthy these days, but that’s partly because I am old now. If I turn rapidly, I’ll likely fall down. :) )
People may be better at mathematics than they realize. Teachers are very, very, VERY bad at teaching it.
> People may be better at mathematics than they realize. Teachers are very, very, VERY bad at teaching it.
I second this. I cringe every time my girls, who are both *very* good at math, say they hate the subject. It has everything to do with the teachers. I think the education program for teachers misses something that must be vital for effectively teaching math, especially in a way that makes kids feel competent and enjoy the subject. At least, that's how it is here in the US.
@hosford42 @chemoelectric @marionline I've always struggled with math, as a blind guy it's really hard for me to visualize, and I don't think school prepared me very well. Just started a CS degree designed for people who don't have much coding experience, and we have to take Discrete Structures. It's definitely a challenge, and I'm not sure that I'll ever "like" math, but I'm just taking it one day at a time and will try to make the best of it.
@ZBennoui@dragonscave.space @chemoelectric@masto.ai @hosford42@techhub.social @marionline@autistics.life Truth to tell, I can't imagine trying to write code while blind. I am massively impressed, and don't understand how you can manage it. I'm very uncertain that I could.
@zakalwe @chemoelectric @ZBennoui @marionline Yeah, I imagine trying to use a screen reader for it, based on my experience with them, and in my imagination I become instantly, profoundly frustrated. I would probably immediately start looking for other options. This is a real consideration for me, as my vision has gotten progressively worse with age and coding is my favorite activity.
@hosford42 @zakalwe @chemoelectric @marionline Yeah having used computers for like 15 or so years, both throughout school and undergrad, it's definitely second nature to me at this point, but it takes me much longer than a sighted person to accomplish a similar task. Case and point, I was in a lab a few weeks ago having to write some Python, and it took me like 30 minutes to write the same amount of code that everyone else wrote in five. No one really knew what to do with me, I was basically on my own lol.
@ZBennoui @zakalwe @chemoelectric @marionline There has to be a better way. I can't help but think that, anytime I hear a frustration like this. There has to be some other way to get information from the computer into your brain faster. Obviously, I don't mean a brain implant made by someone oh-so-trustworthy like Musk. But there has to be some kind of gadget that could improve that process. The tactile screen I described in another branch of this thread would be an option, if we only knew how to build one.
@hosford42 @ZBennoui @zakalwe @marionline
Due to chronic hand pain, for coding I have to input with one of these:
https://en.wikipedia.org/wiki/DataHand
It is still quite uncomfortable.
For ordinary language I can use an Android phone and a stylus.
I started coding in the late 1970s. But my major was electrical engineering and so VERY heavy on the analytic geometry. Plus I did most of my electives in numerical analysis with the math department. Four semesters of numerical analysis.
@hosford42 @ZBennoui @zakalwe @marionline
So I observed a lot during all that.
One of the problems, for instance, is that most teachers make the tests harder than the homework.
Tell me, in what universe does this even SEEM to make sense? Just think about it.
Not that I believe in competitive grading, but assume it exists. In what universe does it even SEEM to make sense for the tests to be harder than the homeworks?
@chemoelectric Most of the tests I gave *were* the homework, which we had just gone over. The sad fact is that few students had actually done the homework, and fewer still paid attention or asked questions upon review. @hosford42 @ZBennoui @zakalwe @marionline
@AlliFlowers @hosford42 @ZBennoui @zakalwe @marionline
Maybe because the only reason they had for doing these things was to get whatever kind of rewards the school offered. Most of them offer insults such as prizes and competitive grading.
I dropped out of high school. Never finished junior year, except technically they passed me through that year. But I actually stopped going. And I have come to realize that half of what they teach in university is actually PLAIN WRONG.
@chemoelectric@masto.ai @hosford42@techhub.social @AlliFlowers@talkedabout.social @ZBennoui@dragonscave.space @marionline@autistics.life Sabine Hofstetter has a very interesting talk on global warming. It's worth going to find it, and watching the whole thing.
One of the most valuable things about it is how she presents it, because she goes through it in successive tiers of education level. "Here's the high-school explanation; and here's why it is incomplete and wrong. Here's the undergraduate explanation; and why it is wrong." And she eventually finishes up with the Ph.D level explanation — which isn't actually that complex or difficult to understand.
In fact, this is the essence of the Ph.D explanation: Atmospheric carbon dioxide heats the earth because it changes the band gap within which radiated heat can escape the atmosphere, which raises the altitude from which heat can escape by radiation to higher levels where the air is colder to start with, which makes it a few percent less efficient for the Earth to radiate excess heat.
And a few percent doesn't sound like much, but you have to remember that it's not a few percent of room temperature, of a few percent of the — let's call it 26°C — difference from freezing, it's a few percent of the 300K difference from absolute zero.
She makes a comment to the effect of "If you don't believe in global warming, it's because you have only a high-school level [at best] understanding of global warming," and what you think you know is wrong.
@zakalwe @hosford42 @AlliFlowers @ZBennoui @marionline
That is much too complicated and also is WRONG.
This is the sort of thing I am talking about when I say that university professors get everything wrong. This explanation is WRONG.
First of all, it is not global WARMING. It is a THERMODYNAMIC CHANGE.
The problem is that Earth used to be a CONDUCTOR of solar energy, but now it is an ABSORBER of solar energy.
That is a fundamental change of mechanism, which requires no graduate school.
@chemoelectric@masto.ai @hosford42@techhub.social @AlliFlowers@talkedabout.social @ZBennoui@dragonscave.space @marionline@autistics.life If you think that this is an example of "university professors get everything wrong", then you have just thrown grave doubt upon your ideas of what is wrong. Because what you just said, and the nonsense you offered in return, sounds an awful lot like "I don't understand this, therefore it is wrong."
@zakalwe @hosford42 @AlliFlowers @ZBennoui @marionline
Uh huh. The old ‘Sit down and shut up, you non-expert’.
And you wonder why people don’t do their homework. Because this is the future they face even if they do their homework--they will be told they are too stupid to think for themselves, and must defer to ‘experts’.
@zakalwe @hosford42 @AlliFlowers @ZBennoui @marionline
I’d love to see an explanation that doesn’t have Earth transforming from a solar energy conductor into a solar energy absorber.
@chemoelectric @zakalwe @hosford42 @ZBennoui @marionline Can y'all untag me in this, please?
@chemoelectric@masto.ai @hosford42@techhub.social @ZBennoui@dragonscave.space @marionline@autistics.life Same. If you think that woo-woo is actually a reasonable scientific explanation, I'm out.