You lost me at stolen training data. You seem like yet another liberal for copyright. That is the primary thesis of your argument. Which is pathetic. I hope you are aware that most of us consider unlicensed usage of training material a feature not a bug. This is home of !piracy@lemmy.dbzer0.com don't act so surprised we aren't morally outraged like you are that someone pirated images or books to use in training data. Many of us would freely do it ourselves. Some of us do do it ourselves.

It's probably a good thing you didn't make an account here. Liberalism isn't tolerated well.

Wow, just wow. I love your comment and fully agree with what you say based on what I know as a software dev with a deep background in math and cumputer vision (my master's thesis was on facial recognition). The thing is that you explain incredibly well. And I had to say that this point you make is specially interesting and is making me think a lot:

"We cannot mathematically prove yet *why* this is the case, but this was practically seen to be the case. I personally suspect that natural language has some deep underlying structure that we cannot see yet, but informs both the AI's ability to learn\*, *and* our ability as humans to learn."

Thanks for sharing your thoughts and knowledge.

How can one of the points from the link I provided to the general list of LLM issues be my main argument? Do you think I'm the one who compiled that list?

And to label me a liberal based on that sole baseless conclusion is absolutely ridiculous. I hate copyright and support piracy just as much as next person.

First, thank you for writing this well thought out reply. I do have some things I'd like to clarify or ask.

We could have absolutely trained these models, albeit at a vastly slower sustainable pace, through the use of distributed and decentralized computing strategies. The issue is that the perverse incentivization structure of capitalism pushed AI companies to grow unsustainably, at great expense to the ecosystem and the people of the world. I.e.: capitalist interests of quick money and infinite expansion triumphed over proletarian interests of survival, dignity, freedom, and sustainability, as usual.

If the issue of resources required nfor training is purely corporate, then why did deepseek use the resources it did for training? It is true they were able to train the model much cheaper than the corporate models, but the scale of the training and amount of resources used was still much larger than one might have expected if the issue was purely a corporate one. Although there is something to be said about geopolitics and Chinese state influence (which is extremely capitalist)

And I also agree that we are seeing a crisis of isolation, but my view is that:

This crisis predates AI.
It is caused by capitalism, specifically the gradual commercialization of previously free or low-cost activities and spaces.
It is exacerbated by ecological destruction and encroachment by the State, which simply makes the world outside your home a worse place to gather.

Totally agree, the issue is not caused by genAI, but this issue is something that in my opinion prevents us from forming the societies we want to live in. It is something I have been giving a lot of thought too before the wide-spread adoption of gen-ai and it is something I keep seeing in the people that use it as a tool to get something important to them done. So many would rather hope an AI translation app works then to ask a friend to help them translate. Or to ask a neighbor to help them with planning a garden. I think using genAI as a convenience limits us from building solidarity with eachother.

It is my personal hope that we live long enough (we as proletarians living in a world doomed by capitalists) so that I can someday train an AI on all of Anna's Archive. I'm sorry if that's offensive to you, but that would be so unbelievably fucking cool and useful.

I'm not offended by this, but I do think the current iteration of genAI not being able to credit creators means that information is siloed and harder to extract. Maybe that changes in the future?

Your understanding of AI systems and their capabilities is probably out of date.

Yeah, I can cop to that. Im not an expert and I don't claim to be.

I think it would help if you tried out the HuggingFace tutorials, or if you pirated Sebastian Raschka's book and did the exercises, to calibrate your expectations about AI.

I've actually ran models locally that I converted to gguf from huggingface before, I've also fine tuned and but a rag system before, but I'll check out the book. Thanks for the recommendation.

Lastly, this comment was written by a real autistic human on a cell phone with zero AI assistance, as are all my comments. Yes I actually write like this 😆

Thank you.

It's a neural network man it's just a computer.

It's just atoms man, just the entropy of the universe.

And to label me a liberal based on that sole baseless conclusion is absolutely ridiculous. I hate copyright and support piracy just as much as next person.

Somehow I'm not convinced because the only ones really relevant to local AIs are the copyright ones, and the other shit is about the AIs hosted and trained by the big tech companies like Anthropic and OpenAI. Like the majority of the shit in here is shit the AI companies are doing and you cite it as a reason why people on dbzer0 using their own home grown or even home built models are bad people?

Don't tell me you aren't trying to do that. You called out Local LLM users in your edit:

EDIT: For those of you who are ready to comment about “wonderful” local LLMs

I don't agree with the original wording of the person you replied to, but i can do my best to explain still.

Firstly, the obvious meaning for the term "AI" here is LLM's, this makes the following points clear:

• Calculators use abstract syntax tree parsing, a small battery, and perhaps a small solar cell, making them efficient for making in-place calculations, completely unlike LLM's which are probabilistic and inefficient for in-place calculations, even disregarding accuracy.
• Books are currently being destroyed in the process of developing LLM's, I'll assume you've heard the news so I'll avoid preaching, but i may elaborate further if you'd like.
• Algorithms are used to give deterministic correct answers, similar to heuristics which give likely-correct answers within specific constraints, both are instruction sets for computers, both are unlike LLM's, which are black boxes, meaning there are no "bugs" ever by definition, and triaging problems boils down to understanding the (absurdly large) training data, avoiding that is the entire reason LLM's were created to begin with.
• the rest are not really that relevant, and I'm sure you can understand the point I'm trying to make.
• Also note: while i make the distinction from LLM's to other forms of machine learning (and the like) they do share some core problems like being black boxes, which have a potential to be very bad in specific contexts in which you need to know why you got a specific answer to a question.

Why is this technology inherently anti-worker and hierarchical?

Anarchism, in all its forms (or at least all that i can think of on the spot), rely mainly on community, inter-personal relationships, and the union of means and ends, LLM's form an incentive to not make connections with other people, to not rely on other people's expertise, and sometimes to even doubt your own.

I'm not even giving this justice as these tools are trained to speak like a person, this has psychological implications, I'm sure you've heard the term "AI psychosis" before, this term may be medically dubious, but it does point at a real concern, this technology already isolates people, the vulnerable of us, and undermines expertise and community.

The way these models are trained, requires consuming (again, absurd) amounts of data, that can only be aquired via exploitation, alienation of labor from worker (as in no work gets properly credited, and that the LLM's are used and marketed as a replacement for actual sources of knowledge and expertise), and destruction of knowledge, as when you replace a source of knowledge, the people maintaining it may choose to stop doing so (via perceived obsolescence).

If we actually want to reduce the demand for corporate LLM's, the answer is not FOSS LLM's, as much as i love FOSS software, this technology only really drives us apart, we need to do it the tried and true method: building community, mutual aid, mutual responsibility, we need to perserve sources of knowledge and expertise, we need to keep expertise valuable, and we need trust, this requires community.

Apology for the length of this, in short: The demand for AI is a spook, go do community work.

Definitely a lot of capitalist fuckery, can't argue there.

I don't have one but if I did I would use it to generate images. Maybe also connect to the AI horde so other people who can't afford their own can get the benefits when I'm not using it.

This one?

Build a Large Language Model (From Scratch)

I am also against genAI.

I've heard many arguments for it, mainly from tankies, but also from leftist spaces and anarchists. The ideological idea seems to be that it is a way to allow everyone to have access to things they weren't skilled enough to do. Something like the "democratization" of labor.

I am obviously against that statement, mainly in 2 fronts.

First, we have skilled labor for a reason. Some people dedicated years of their life to perfect a skill or a trade, and they are trustworthy experts with experience in the matter. It would be unfair for those people if they got replaced by a sloppier version of themselves, after spending so much time and effort into their skills.

I would also not trust a machine like I would a human, because the machine has no experience and can't make decisions. We're removing the human value of a skill, while also reducing its quality, when we could promote collaboration between humans, and make them feel valued while getting a better result.

Second, AI was built on stolen labor. Even if you use a self hosted model, running on old hardware that was thrown away, using solar and wind energy, its usage is still morally wrong.

The truth is, all the current models were trained on databases of data mined without consent, in many cases to the detriment of its providers.

On top of that, even if the models are used locally (and even if they are tuned locally), the initial training happens on massive datacenters that consume lots of resources. And what you get isn't the first attempt at making a model. For every successful model out there, thousands of other models were trained and abandoned.

AI is also currently making us dumber, and multiple studies have already shown it. A lot of pro-AI people have already become dependent on it, and because of that they will defend it. Even if it would have been against their morals. It's normal, we as humans tend to get defensive in those cases. But it means a lot of pro-AI discussion in leftist spheres is driven by that.

I think we should be really careful of AI because of that. AI is like a drug, and once you become dependent, you can't easily go back. Our brains are wired to offload their thinking to AI, and once they do that, picking it back up is a big effort.

I also haven't gone into the artist rabbithole. Of course, genAI harms artists. All kinds of them. Writers, painters, photographers, musicians, etc. People are replacing human art with synthetic slop, to the detriment of artists.

Sometimes it's because they don't know better. Sometimes it's because they feel like they couldn't be able to create something without the "help" of AI. But art is not the end result, art is the process. And by using AI, you're destroying the process to end up with and end result.

I feel like the same arguments above apply to genAI usage for "art". People are replacing artists with the lower effort and lower quality versions of them, which wouldn't even be possible without stealing the art of those artists in the first place.

This comment is already long and ranty enough, and it's late at night, so I don't expect it to be very coherent. So I won't go deeper.

However, as an aside, I'll mention that I've been blanket banned before from communities, genAI related and not, by MysticMushroom, for making anti AI comments somewhere else. I have asked another mod for an unban in a non genAI related community ( !europe@lemmy.dbzer0.com ). Afaik MysticMushroom unbanned me after asking, so credits for that.

Coding assistants and porn are the most common uses.

With A charitable interpretation I think that comment was just an off-hand insult to the rich and a way to make people think who they're "supporting". It was not meant to label users of genai as Pedos.

only ones really relevant to local AIs are the copyright ones

No, it is about lack of consent. The open models are all trained through non-consential heavy web scraping, killing off smaller web servers by the thousands and driving server admins to despair by their use of (ususally non-consential) residential proxy bot networks.

The total lack of respect of any consential boundaries is the real problem. It is a pure strawman to claim that this is about copyright.

If the issue of resources required nfor training is purely corporate, then why did deepseek use the resources it did for training?

So first of all, there is a difference between using X total resources over three years versus using X total resources over three decades. A more extreme example is the difference between burning a full tank of gas over a few days of driving versus burning all that gas at once by lighting it on fire.

China's contributions to the AI race are, in my view, tarnished by many of the same problems as Western AI systems, since China is (state) capitalist. (That being said, Chinese companies do usually make their models weight-available, which is commendable, but not enough.)

While I suspect that we have not thrown enough theoretical math at the problem of LLM training (we have thrown lots of empirical math at them; useful for guiding the development of theory, but not theory itself), I also suspect (frankly based on my limited experience training and using these things) that the useful structural properties of LLMs only manifest themselves when you train with large datasets.

So the problem of large datasets is, in fact, an inherent part of the technology. My view is that we have to tackle these things head on.

1. Distributed computing. Internally, LLM companies are computing at such a scale that they already must use distributed computing technologies to spread the load out amongst many machines. My vision would be something like BOINC, where people and collectives who benefit from AI systems can give limited access to the computing power of their systems. Note that AI Horde does something like this for inference. Hence why it is so important to seize the means of production from the capitalist class. Not ask, take! They have the tools and power to do this sustainably, but they chose to do it wrong so they could make a quick bag.

2. Leverage existing resources sustainably. The logic here is that individuals and collectives spreading out the effort won't necessarily save resources or energy integrated over the whole planet. In fact, as a planet, we might spend more energy on a geographically distributed system if it is done at the scale AI companies are doing it. However, the issue is not just "we're spending too much energy "; it's "we're spending too much energy, too fast, and we are overloading and obliterating local ecosystems, and we are doing it in a way that exacerbates existing climate". I.e., it's not just quantity, but also space and computing time whose distribution is mismatched.

Like what the capitalists have pushed on us in basically three years...it really should have taken at least a decade.

So many would rather hope an AI translation app works then to ask a friend to help them translate.

Depends on what I need help translating. Basic Spanish? I'll ask my mom. Farsi? I don't know anyone personally who speaks Farsi, to the best of my knowledge. Advanced Spanish? Ain't no way my mom's gonna beat a machine, even a pre-2023 AI translator. I could go hire a translator, but I suspect that they have more important things to translate. I.e., translation is usually a routine task, and human translators can show up for the fun stuff (translations that require human supervision).

Or to ask a neighbor to help them with planning a garden.

I wouldn't ask AI personally. I would probably look on LibGen for botany books, or drop a post on Lemmy since I'm sure there are a few botanists lurking around.

I think using genAI as a convenience limits us from building solidarity with eachother.

It can, but it doesn't have to. And I think that a lot of those examples are people misusing the tech. In particular, people are using the tech to "solve" problems with simple, well-understood solutions. LLM systems are suited for problems without algorithmic solutions, where either a partial and/or approximate solution is better than none at all.

I do think the current iteration of genAI not being able to credit creators means that information is siloed and harder to extract. Maybe that changes in the future?

I would argue that at least for math, you can actually get really far by telling the AI to cite its sources in the system prompt and giving the harness access to your PDF library (I do this in a virtual machine with a subset of my actual library that I have backed up in case I lose it). (And of course you triple check the citation it gives you.)

Also, the thing about math in particular is that we do not particularly disagree with the capitalists about mathematics. We all agree that 2+2=4 and the indefinite integral of cos(t) is sin(t) + C. And we also widely agree on how to typeset mathematics, namely "rewrite unreadable spaghetti code in LaTeX until the rendered PDF looks gorgeous." So a LLM with a capitalist bias... isn't really a problem for math typesetting questions. Conversely, I would never ask a capitalist LLM to, say, summarize the contents of Mutual Aid by Kropotkin, even if I give the system the PDF. (Most capitalist LLMs probably do recall the broad contents of Mutual Aid and the most important texts in anarchist theory, simply because they sucked up all our literature too.) I think there are other walks of life where anarchists and other class-conscious proletarians do not disagree with the capitalists, although personally my disagreements with capitalists are extremely broad, almost universal, so I understand if people have trouble finding common ground with them!

Your understanding of AI systems and their capabilities is probably out of date.

Yeah, I can cop to that. Im not an expert and I don't claim to be.

That wasn't just aimed at you ... that holds for everyone, myself included. Again, the capitalists are overselling the convergence to AGI specifically...but things are moving incredibly fast. So if you're not using the systems regularly, it's honestly going to be very difficult to keep up with the development of the tech. (I'm not saying that you should necessarily start using AI systems regularly just to keep up; it will suffice to just remember that your knowledge might be out of date by the time you finish your argument.)

You haven't heard people whine and complain about art theft as much as I have. I agree with what you're saying about large companies scraping data but you are being too charitable to these people. People do indeed bitch and moan about art theft and I have no tolerance for that level of liberal apologia.

I hope not but you never can tell with them.

You're very welcome, and thank you for reading!

cumputer vision

Please I beg you do not change this typo 😆

... unlike LLM’s which are probabilistic and inefficient for in-place calculations, even disregarding accuracy.

Correct, although why you would want to use an LLM for in-place calculations is beyond me. That's the wrong tool for the job. I also think using hammers on screws is a bad idea, but I don't see how that makes hammers "a tool of the Epstein class"

Books are currently being destroyed in the process of developing LLM’s, I’ll assume you’ve heard the news so I’ll avoid preaching, but i may elaborate further if you’d like.

Sure, but that's a capitalism problem, not an AI problem. Thats about artificial scarcity, not statistical probability calculations.

Algorithms are used to give deterministic correct answers, similar to heuristics which give likely-correct answers within specific constraints, both are instruction sets for computers, both are unlike LLM’s, which are black boxes, meaning there are no “bugs” ever by definition, and triaging problems boils down to understanding the (absurdly large) training data, avoiding that is the entire reason LLM’s were created to begin with.

Uh wut? There are absolutely bugs. I don't know of anyone who has ever said that LLMs don't have bugs.

the rest are not really that relevant, and I’m sure you can understand the point I’m trying to make.

No, actually I don't. Please explain further

LLM’s form an incentive to not make connections with other people, to not rely on other people’s expertise, and sometimes to even doubt your own.

I could say the same thing about calculators, or waterwheels, or books. Do you think those things are "inherently hierarchical and anti-worker" ?

I’m not even giving this justice as these tools are trained to speak like a person, this has psychological implications, I’m sure you’ve heard the term “AI psychosis” before, this term may be medically dubious, but it does point at a real concern, this technology already isolates people, the vulnerable of us, and undermines expertise and community.

Yeah, a powerful tool used for evil goals is going to have an evil outcome. AI psychosis is not inherently a part of LLMs. That's how they are specifically built in order to make people use them more. Once again, this is a problem with capitalism.

The way these models are trained, requires consuming (again, absurd) amounts of data, that can only be aquired via exploitation, alienation of labor from worker...

That's not true at all. Many of them are trained on data that has been made freely available by the creator of the data.

...(as in no work gets properly credited, and that the LLM’s are used and marketed as a replacement for actual sources of knowledge and expertise)...

So, like books, and online videos, and comics, and articles, and blogs, etc? So do you think those are also "inherently hierarchical and anti-worker" ?

...and destruction of knowledge, as when you replace a source of knowledge, the people maintaining it may choose to stop doing so (via perceived obsolescence).

Again, wut? Do you think books are a destruction of knowledge because they replace scrolls, or direct conversations from the writer? I feel like I really don't understand what you mean here.

There are plenty of them that are not trained through non-consential scraping. Just because some of the people making AI models are assholes don't mean everyone making AI models is an asshole.

How are they fundamentally fascist? They don't exist for those purposes, but they can be used for those purposes.

That's like saying "hammers exist to beat people's heads in". Yes, they can do that, but that's not what they are meant for.

Name one that is actually useful 🤷

So it is fine to ignore the consent of artists? Sure, they usually talk about copyright, but what most mean is consent and not the legal construct.

Also, "bitch and moan" is pretty mysogynist language, especially in this context.

First, we have skilled labor for a reason. Some people dedicated years of their life to perfect a skill or a trade, and they are trustworthy experts with experience in the matter. It would be unfair for those people if they got replaced by a sloppier version of themselves, after spending so much time and effort into their skills.

Is it "unfair" to bakers that I make my own bread at home with a bread machine? What about using a calculator? Or a dishwashing machine?

Second, AI was built on stolen labor. Even if you use a self hosted model, running on old hardware that was thrown away, using solar and wind energy, its usage is still morally wrong.

Well, it's not all built on stolen labor. A lot of it is built on data that was given away freely. You also didn't really provide a reason why it would still be morally wrong.

On top of that, even if the models are used locally (and even if they are tuned locally), the initial training happens on massive datacenters that consume lots of resources. And what you get isn’t the first attempt at making a model. For every successful model out there, thousands of other models were trained and abandoned.

They don't have to use massive resources, we just choose to do them that way because of capitalism. Cities use massive amounts of resources, and many of them aren't successfull on the first try. Should we stop building cities?

The rest of your post is more of a complaint about capitalism, and misunderstandings of how we choose to make AI models, not how they inherently have to be made.

A corporation known for supporting apartheid creates a product. The product doesn't loudly proclaim its support, paying for the product puts money into the corporation's pockets.

That's like the argument that "guns don't kill people" - what the fuck else are they for?

Including after never interacting with them. Fucking sloppers man istf, they're so conditioned by the mechanical sycophant that just letting your gaze wander by them without getting on your knees and tearing their pants off is a grave insult. They seriously aren't even people.

A good Master reduces demand for less beating on the plantation

Thats the one

Covers a basic classification model and assistant model. You'll want some understanding of deep learning and python (its not too heavy, but you should have a decent grasp of the concepts and basics of python to understand what's going on).

Edit: On the easier side, you can do something like unsloth and modify a base model. Easier, but heavier requirements depending on what you start with. Good for testing concepts, but highly specialized models are the superior approach over general purpose IMO.

The sloppers are gonna delete this. I posted something really similar earlier. Don't talk to sloppers like they're people. They're not. Their brains are rotted and they're incapable of honesty. Online they are very similar to fascists, though differences do come up in person.

You want me to do a basic internet search for you?

Fine: https://www.fairlytrained.org/certified-models

A cursory look shows that those are either toy models with no real usability and/or closed, so not at all what we are talking about here. Try a bit harder, because so far you are just proving my point.

If LLMs were only made by apartheid supporing corporations, you might have a point. But many different people and groups create LLM's for all kinds of things: Statistical analysis, generating stories and characters, coding assistance, design assistance, process assistance, language translation, rubber ducking, debate assistance, running D&D games and many more.

So you admittedly didn't even look at the models, nor their uses, but have determined that they don't match your criteria?

If you aren't going to accept a response just because you don't like that it contradicts what you want to believe, I'm not going to waste my time engaging with you.

You're right. I'm sorry about that. I should have known to avoid language like that. I changed it to something better.

As for the other thing though. I'm not on board with that either. That's just a socially enforced version of Intellectual property. It all converges on the idea that people are allowed to own intangible stuff like ideas, or bits. It's the same liberal idea and expectation as copyright law just with different enforcement. Not so different though, they both expect blind compliance and assume moral superiority.

It is unfortunately the most common response from anti-AI people. They ignore everything that contradicts their opinion and beliefs because what they want isn't to seek a better understanding, they want other people to see their opinions as facts.

I seem to have looked at the models more than you did. Which one of those has open weights and is more than a toy model?

Uff, that is the clearest case of projection I have seen in a long time 🙄.

I am not "anti-AI", I have run local models on my own hardware and are pretty aware of what is available and possible with them. But every time I point out the clear technical and moral issues with them, some AI-booster comes along, points to something that they clearly didn't bother to even look into themselves and then ignores everything that contradicts their opinion and beliefs...

So you would be ok with sharing nude pictures of a person without their consent? The photos are just intangible stuff, right?

That's got to be the most disingenuous bad faith comparison yet, comparing art piracy to exposing someone's nudes. No asshole, sharing nudes or personal information is not the same as pirating your art.
@Flatworm7591@lemmy.dbzer0.com What do you think? Do you think this is as bad faith and disingenuous as I do?

In any case we're done here. It's clear you're not interested in real discussion.

I read every blathering word and nearly started my reply with "Gish gallop."

If you don't care when some of your objections are addressed, you don't mean anything by them.

"Everyone I disagree with is a capitalist" and other arguments by the mentally numbed.

disengage

You really think it is morally ok to pirate art against the explicit request by an artist to not do so? Like the artist personally talked to you and asked you not to do it? If you really think so than you have serious issues with respecting consent and are violating core anarchist principles. The nudes example is just the same, but more obvious to get the point accross.

Thank you for this! It really put a lot of my thoughts into words along with some deeper insight. I think if we are going to push for a better society then we need to pay attention to people like you who are experts in the field and can really tease apart the pros and cons of things like this.

I see a bit of a panic in current anti-AI sentiment at the moment, and honestly rightfully so, as it is being weaponized against us. But to outright dismiss it or to try to put the genie back in the bottle could be shooting ourselves in the foot. If we can take control of it ourselves and push out the capitalists we could have something that is genuinely useful, and most importantly, ours.

Ai isn't killing the planet, people are

Putting shit on the web is giving consent to scrape it

What