Little bit of everything!

Avid Swiftie (come join us at !taylorswift@poptalk.scrubbles.tech )

Gaming (Mass Effect, Witcher, and too much Satisfactory)

Sci-fi

I live for 90s TV sitcoms

  • 1 Post
  • 64 Comments
Joined 1 year ago
cake
Cake day: June 2nd, 2023

help-circle








  • It’s happened a few times in my career where people tell me I’ll be obsolete, but it’s always been some company hyping their new product and suits frothing at the prospect of not having to pay me anymore.

    So far they’re like 0 for 8 or so.

    Now I will say the goalposts move. What I’m doing now is for sure not what I was doing 10 years ago. I’m definitely heavier in devops and infra than where I was before (ironic because they said we’d never have to worry about that stuff again if we moved to the cloud). AI is still basically machine learning, just in a while loop, so I’ve spent time learning that. So, in a way, yes we’re obsolete in the sense that if I was the same engineer I was 10 years ago I wouldn’t be worth nearly this much, I had to grow and evolve with technology.






  • Okay so I had a meltdown last year. I was staring down a startup that was circling the drain, I knew my time there was limited, and I was being bombarded daily with layoffs and friends not being able to find work, while hearing constantly that I was going to be left behind due to AI. (of course the layoffs were happening because tech CEOs heard AI and started frothing at the idea of getting rid of some of their most expensive staff)

    So, I took it on myself to learn AI. I figured well, if it’s coming for my job I might as well learn how it works. And oh lorde, did I learn a lot. To the point where I’m running several LLMs now at home, I have them running in k3s, across multiple servers, and have built several apps to interact with them. I’ve trained finetuned LLMs, I’ve played with image generation, voices, I dove headfirst in. Eventually I did lose that job, and that gave me a couple months to focus even more before finding my current one.

    My biggest learnings, which I’m sure many of you know:

    • AI is a very neat technology, and it has several real applications, but those applications are extremely limited by the limitations of AI
    • LLMs and AI are incredibly hard to control. You can’t just say if(nsfw) dont(). You have to spend a lot of time forcing the LLM to not give weight to the users course, and it’s to the point that it hardly seems worth it.
    • Companies love the idea of LLMs for their chatbots, but using the above it’s incredibly hard to prevent the chatbot from doing it’s own thing. You can say “We only have a return policy of 14 days” but if someone works at it hard enough and tells the LLM that it can still perform the return because they say so… is it really that useful? LLMs have no hard rules
    • LLMs have very real hardware restrictions. At home, it’s a single GPU. In the cloud they have some clever tricks to share memory, but overall it’s still mostly limited by the GPU. We’ll see as we’re moving forward what shenanigans NVidia comes up with, but LLMs and AI are essentially a brute force approach, and we can see that in how much power they soak up. You can see ChatGPT slows down once your conversation goes on long enough, it’s running low on memory
    • AI is not new. It’s still ML under the hood, it’s just coming up with unique ways to reuse it. Again with the brute force, I didn’t realize that for every token (word for simplicity), You’re entire conversation is passed in to the model, in which it will spit out one more word. Repeat for every word. That’s all it is. It’s just predicting the next word. Image generation is just predicting the next pixels, and then loops around again until it comes back. There is no consciousness, there’s no real nuance to it, that’s it. A predictive engine surrounded with a while loop.

    There’s more but this is too long already. It’s neat, it’s useful, but the hype was just as intense as blockchain. We’re going to see some real great usages out of it, like integration with something like Word or a browser to summarize things is honestly a good idea. But there are so so so many pitfalls.

    For coding? I think it’s a great place to get started, or to get an idea. I would never trust it in production. It will take a very long time for us to get to the point where you can say “Go build this feature” and I would blindly trust what it generated.






  • Nah, I don’t buy that. When you’re in critical infrastructure like that it’s your job to anticipate things like people being above or below versions. This isn’t the latest version of flappy bird, this is kernel level code that needs to be space station level accurate, that they’re pushing remotely to massive amounts of critical infrastructure.

    I won’t say this was one guy, and I definitely don’t think it was malicious. This is just standard corporate software engineering, where deadlines are pushed to the max and QA is seen as an expense, not an investment. They’re learning the harsh realities of cutting QA processes right now, and I say good. There is zero reason a bit of this magnitude should have gone out. I mean, it was an empty file of zeroes. How did they not have any pipelines to check that file, code in the kernel itself to validate the file, or anyone put eyes on the file before pushing it.

    This is a massive company wide fuckup they had, and it’s going to end up with them reporting to Congress and many, many courts on what happened.