It is actually not so difficult to see this for yourself in a much simplified setting. One can easily build a “Small Language Model” that extracts correlations between only three consecutive words. On the web there’s plenty of short scripts that do this; here and here is one example. The output created by such a SLM can have remarkably long sentences with grammatical meaning (see the examples in the links above); this is remarkable since all it learned was correlations between triplets of words.

Now you can take a large amount of output from such a SLM, and use it to train a second, identical or even better SLM, then check the output generated by this second one. You’ll see that the new output is less coherent than the one from the first SLM. Give the output of the second SLM to a third, and you’ll see even less coherent text coming out. And so on.

They aren’t out of context, and you have just said the same thing. Data processing can help in removing noise, but it can’t help in creating information or extracting information that wasn’t there in the first place. In fact – again as you said – it can end up destroying part of the original information.

LLMs extract word correlations from textual data. Already in this process they are losing information, since they can’t extract correlations beyond a certain (yet large) length, and don’t extract correlations at shorter lengths. And in creating output they insert spurious correlations that replace (destroy) some of the original ones. This output will contain even less information than the original training data. So a new LLM trained with such an output will give back even less.

Yes it does. Indeed it is a mathematical theorem from Information Theory, called the data-processing inequality. Quoting from two good textbooks on Information Theory:

“No clever manipulation of the data can improve the inferences that can be made from the data” (Cover & Thomas, Elements of Information Theory §2.8).

“Data processing can only destroy information” (MacKay, Information Theory, Inference, and Learning Algorithms exercise 8.9).

You’ve read the stances of all different people. I agree with most and I’m a bit more conservative: I switch to a LTS (even-numbered) release only when its main non-LTS (odd-numbered) upgrade is out; and skip all non-LTS.

During our in-person visa appointment in Seattle, a shooting involving CBP occurred just a few parking spaces from where we normally park for medical outpatient visits back in Portland. It was covered by the news internationally and you may have read about it. Moments like that have a way of clarifying what matters and how urgently change can feel necessary.

Our visas were approved quickly, which we’re grateful for. We’ll be spending the next year in France, where my wife has other Tibetan family. I’m looking forward to immersing myself in the language and culture and to taking that responsibility seriously. Learning French in mid-life will be humbling, but I’m ready to give it my full focus.

Sounds like a splendid person.

It’s also a smart move considering that, with age-verification laws advancing, it looks like a good part of the Linux world will become with time another instrument of mass surveillance.

Still more acceptable, in my opinion, than going from “using” to “leveraging”…

💪 🤝

It is an ultimatum, nobody said it isn’t. So what? An ultimatum, just like an argument between people, can still be polite and respectful, as opposed to rude or threatening.

Here’s an example of a sentence that isn’t an ultimatum and is non-polite and non-respectful: “Go and read a dictionary and think with your own head, instead of babbling about LLM output”.

Thank you. My initial thought was simply that we users should tell how we feel to the FOSS and Linux developers of the software that we use and are especially attached to; but we should do it in a polite way. I’ve now realized that “FOSS” does not have the connotations that I thought it had, like “community-oriented”, “inspired by human rights”, and similar. My bad, honestly. There clearly are developers for which working with FOSS is really not different than working in or for some corporation. But luckily there are also developers for which FOSS does have those extra meaning. What’s important for me now is to keep supporting the latter, and ignore or shun the former.

Yes, it’s polite, as opposed to rude. Go and check the meaning of “polite”. One for example says “She politely asked them to leave”.

I don’t think enough developers realize that the majority of users does not want this. They’re acting exactly like the legislators: “we don’t give a shit about what the people think”.

The legislators won’t take the Linux community seriously, because the developers aren’t taking the community seriously either.

I don’t think it’s unproductive at all. Positive changes and resistance to negative changes are caused by many, extremely different and complex factors, one of which is voiced discontent at all levels. It’s a chain of pressures. Some elements press on other elements which are not the final target, but this pressure makes them in turn exert even more pressure closer to the target. Edit: take for instance the Montgomery bus boycott – was the bus company responsible for the law? shouldn’t the black people have done the boycott then?

Without such internal pressures, positive changes may fail. History shows examples over and over (a good read is the historian Barzun’s From Dawn to Decadence). I’m sure that if it wasn’t me posting stuff like this, it’d be someone else, and maybe sharing a much less polite post.

Also, I think that this kind of moderation ends up giving a very false picture on the forums, as if everyone discussing there doesn’t really mind about the topic.

I think developers can do something about it. They don’t want to, maybe for obvious reasons, and I respect their choice. But there is a choice.

If you visit, say, the website of the American Association of Physics Teachers or similar associations, you can read their complaints, protests, and alarming messages about cuts and bad policy changes in the education system. So clearly laws like these are not truly targeted at the well-being of children. Fu*king invest on education of children and adults instead.

We forget that there’s no just law, there’s also morality. And sometimes they are against each other. And we must make a choice.

I don’t live in California, but I was one of the many who wrote to EU parliament members against the chat-control law proposals, and participated in other local activities about that.

I’d be happy to write to the lawmakers in California. I suppose my email would immediately go into their spam folder.

No, because the Afghan woman doesn’t live in California…

No official decisions yet, although in a discussion thread I read developers basically saying “the law is the law”. This is why I politely wanted to let them know, as a KDE user, what’s my stance.

“Leveraging”

In principle I agree with you, pacific discussion and democracy should be the way to go. But it seems that “discussion” doesn’t lead anywhere these times. Politicians do whatever they like (or what lobbies tell them to do), without checking if the majority of the population really agree with some decisions. A developer does whatever he likes, without bothering about the more or less pacific feedback he gets on github. Nobody really seems to want to have a discussion. Well guess then what the “mob” does at some point: they don’t care about discussions anymore either, and they do as they please too.

I fear that riots will start on a larger scale. Even if the context today is different, the situation reminds me somewhat of what happened with the 1981 riots in Toxteth, in Brixton, and other previous riots. Unjust or misused laws; deafness of authorities about discontent; innocent and not-so-innocent people getting hurt.

Of course there are no obligations and he’s’free to do as he pleases. Likewise, the community or I are under no obligations of not criticizing him for what he chose to do.

He did not just suggest it. He went on and implemented it. All while the community was telling him “we don’t want this”, “stop with this” – look at the comments on GitHub. Yet he neglected all this feedback.

As an open-source volunteer, you work for the community, right? If you go ahead while the community is telling you “we don’t want this”, then whom are you working for?