I mostly share Josh's opinion, but I think a lot of these posts that talk about Senior vs. Junior experience when working with AIs is kind of rubbish. Sure, you get better results as a Senior working with AI tooling and struggle more as a Junior. Nothing has changed in that equation except the amplification.
What folks seem to avoid is that a Junior (in ANY subject) has the ability to LEARN so much faster with an AI research assistant, and that becoming an expert has accelerated for those with the personal stamina to dig deep (this as a requirement hasn't changed). I spend just as much time with my AI tooling asking questions as I do asking it to "build" or "fix" things. "How does this work?". "Can you suggest other tools?".
I think some people always think about AI as an input / output relationship, when a lot of the time, the fiddling in between, with or without AI was always the important part. Yes people will suck in the beginning, against they always did. I think the good folks though will suck for a MUCH shorter time than I did getting into things.
A lot of people will drop out and get discouraged. That happened before too. Learning things requires persistence. I think the only real case to be made is that AI's sense of immediate pleasure can neuter people away from running into friction. AI natives likely won't understand friction and question it.
> a Junior (in ANY subject) has the ability to LEARN so much faster with an AI research assistant
This is a testable hypotheses with severe lack of citations. Intuition would argue the opposite. We learn by using our brains, if we offload the thinking to a machine and copy their output we don‘t learn. A child does not learn multiplication by using a calculator, and a language learner will not learn a new language by machine translating every sentence. In both cases all they’ve learnt is using a tool to do what they skipped learning.
This seems to me like one of those things where people go into it with widely different initial assumptions.
1. AI is for cheating and doing the work for you. Obviously it won't help you learn faster because you won't have to do any thinking at all.
2. AI is an always-available question answering machine. It's like having a teaching assistant who you can ask about anything at any time. This means you can greatly accelerate the process of learning new things.
I'm in team 2, but given how many people are in team 1 (and may not even acknowledge team 2 as even being a possibility) I suspect there may be some core values or different-types-of-people factors at play here.
This is also a testable hypothesis. I would like to see usage statistics before making assumptions here but my gut feeling is that an overwhelming AI usage (like > 90%) would fall into your category 1.
But even with category 2. I think that still does not absolve AI as a cheating machine. Doing research is a skill and if you ask AI to do the research for you that is a skill a junior developer simply never learns.
"The expertise reversal effect is present when instructional assistance leads to increased learning gains in novices, but decreased learning gains in experts."
There's a whole lot of depth to the question of how AI tools support or atrophy learning for different levels of expertise.
Actually, you're both right. Using AI as a supplementary learning aid -- i.e. students use AI as a personalized tutor but still do the assignments themselves -- produces better outcomes. But using AI as a crutch -- i.e. using it to do the assignments -- produces worse outcomes.
> students use AI as a personalized tutor but still do the assignments themselves.
So your first study actually concludes the opposite. It concluded that all AI users performed worse, but the effect was smaller for students which used AI as a tutor.
The second meta analysis I don‘t quite understand. I understand they conclude that using AI tutor shows significant improvement, but I don‘t understand the methodology. I may be misunderstanding but it seems to simply count papers which shows positive outcomes and reaches conclusion that way. I think that methodology is deeply flawed as it will amplify whichever biases are present in the studies it uses. I also think the lack of control groups is a major issues. If we are comparing AI tutor to nothing, off course the AI tutor is gonna perform better. We need to compare to traditional methods. And this is especially relevant in our discussion because junior developers usually have excellent access to senior developers (via peer review, pair programing, etc.), much better then student’s access to tutors for that matter.
> The results indicated that students employing AI tutors shown significant improvements in problem-solving and personalized learning compared to the control group.
Now when I look at the control group it claims this (also in the abstract):
> Participants were allocated to a control group receiving conventional training and an experimental group utilizing AI technology,
But when I look into the methodology section I see this:
> The researchers classified the patients into two groups: MathGPT and Flexi 2.0
MathGPT and Flexi 2.0 are both AI tutors. Now I am confused, where is the control group and how was this “conventional training conducted”?
The methodology section actually tells a different story from the abstract:
> This research utilized a quantitative methodology via a quasi-experimental design.
By quasi-experimental design they mean that they tested the same students before and after AI intervention. And concluded that the AI tutor helped them improve. Now this is not what control group means, so the researchers are actually lying by omission in the abstract. This is a spectacularly bad experimental design and I wonder how it would pass peer review, so I look at the publisher Jurnal Ilmiah Ilmu Terapan Universitas Jambi. So not exactly a reputable journal.
I still stand by my no evidence for a testable hypotheses. I suspect that your first link is actually correct in that AI is bad for students and just less bad if it is used as a tutor.
I hadn't looked at that study you selected, but yeah the methodology conflicts with the abstract (Also it low-key seems to be an ad for "Flexi 2.0.") It does seems to be a shady paper, with a small N and in a journal of questionable repute.
That said, there are 80+ other studies listed in the meta-study, which is pretty frank about its limitations. (Note the snippets about positive biases in the conclusion.) It is going more for quantity over quality and is transparent about the statistical findings of each one (or lack thereof; see the count of "Not reported"s.) All these references have a myriad of results, but across the spectrum of well-designed studies at reputable venues to the other end, they follow the same themes, so I don't think this can be dismissed that easily.
But if you want, here's more research (some of which I linked in a sibling comment https://qht.co/item?id=48241839) which has similar findings:
This was the only study from the meta analysis that I read, and I picked it because it made the strongest claim out of all of them.
This is in the opening of the results section in the meta-analysis:
> In the final screening phase, a rigorous full-text analysis evaluated the methodological robustness and empirical validity of the remaining studies. [...] The final corpus comprised 88 studies that demonstrated robust empirical evidence for LLM applications in educational contexts.
The inclusion of the study I read does not give me confidence that this statement is true. And the fact that they reach their conclusion by simply tallying up the positive vs. negative studies makes me conclude that this meta-analysis is practically useless. They do admit this in the conclusion (which is probably why it passed peer review [assuming the peer reviewer didn’t read the same citation as me as I am 100% certain they would have asked for it to be excluded]). But that pretty much just leaves us with nothing. We are exactly where we started. No evidence that LLMs help students beyond traditional methods.
Now I am not gonna read that Anthropic study. It reminds me of Cigarette companies finding the health benefits of cigarettes. That leaves that excellent 3-study review. In their first study they found LLM has negative effects on students (in line with the first link you showed me). In the second study they found no effect. And in the third study they found mixed (nuanced) effect where using LLMs as tutor helped students in one aspect but had negative effects on others. This is by far the best study you have presented me but it still does not change my opinion. There is little evidence that LLMs (even when used as a tutor) help people learn better traditional methods.
What makes me even more against this sentiment is this quote from the conclusion of the 3-study review paper:
> Our results suggest that students prefer to use LLMs to substitute rather than complement learning activities.
So on their own, students are more likely to use LLMs in a way which is harmful to their learning. I would expect similar behavior of junior developers.
As a precondition I think we have to assume that the person in question 1) wants to learn and 2) is smart enough to absorb new info and apply it and 3) reflects enough to adjust their approach when hitting bottlenecks or making mistakes 4) has a drive to create. Without these, self driven learning is not viable - and that has very little to do with AI.
For such a person, I believe AI can be very empowering for learning. Like Google, wikipedia and stack overflow, Arxiv before it - AI tools give access to a lot of information. It allows to quickly dig deep into any topic you can imagine. And yes, the quality is variable - so one needs to find ways to filter and synthesize from imperfect info. But that was also the case before.
Furthermore AI tools can be used to find holes in arguments or a paper. And by coding one can use it to test out things in practice. These are also powerful (albeit imperfect) learning tools. But they will not apply themselves.
Who is talking about self driven learning? Every workplace teachers their juniors how to do their job, and how to become better at their jobs.
And as we are talking about junior developers it is safe to assume your conditions (1), (2), and (4) are all true, if any of them are false, then why did that person apply for and get a job as a junior developer? As for condition (3), all workplaces eventually hires a person who does not fulfill this, then they either fire that person, or they give them a talk and the developer grows out of it and changes their behavior to fulfill that condition.
Aside: you listed 4 conditions for learning. I am not sure these are actually conditions recognized as such by behavior science. In fact, I doubt they are and that these conditions are just your opinions (man).
I think the key word is ability, and I fully agree with that. Using GenAI as a teaching aid can supercharge learning, especially as it makes it very easy to learn by doing. The problem is that people use GenAI to do and hence don't learn.
(The preliminary research so far supports this: using AI to do the hard assignments produces poor learning outcomes, but using AI as a tutor, or even just for help with the hard assignments, produces slightly better learning outcomes.)
I think what you're seeing is the effect of the incentives of the system. The system uses simplistic numbers like grades as proxies for actual learning, and these grades heavily influence students' job prospects, and so you're simply seeing Goodhart's Law in action. Given how easy current methods of skill assessment are to game with AI, my guess is the entire system has to be overhauled.
> using AI as a tutor, or even just for help with the hard assignments, produces slightly better learning outcomes
Source? The few people I’ve seen try to do this wind up with a terrible understanding of the material, with large knowledge gaps and one or two fundamental fuckups. In every case, an introductory textbook would have been better. (It would also have been harder.)
For coding specifically (there are many studies out there by now, but I know of these offhand):
https://www.mdpi.com/2076-3417/14/10/4115 -- probably the earliest one of its kind, finds over-reliance degrades critical skills but supplementary use is mostly harmless.
https://arxiv.org/html/2601.20245v2 -- Anthropic's study, same as above except supplementary use (like clarifying concepts) can actually be beneficial.
https://scale.stanford.edu/ai/repository/ai-meets-classroom-... -- "Students who use LLMs as personal tutors by conversing about the topic and asking for explanations benefit from usage. However, learning is impaired for students who excessively rely on LLMs to solve practice exercises for them and thus do not invest sufficient own mental effort." Interestingly, they found simply disabling copy-paste on the chatbot interface resulted in better outcomes!
(Multiple studies find that the outcome depends on how AI is used. Surprisingly, incorrect guidance / unreliability / hallucinations appear to be a bigger problem than over-reliance! That could also explain poor performance in some cases.)
My intuition, supported by these studies, is that as long as students are willing to do the hard cognitive work -- for which there is no substitute, really -- having LLM assistance is a boon. Which makes sense, it's comparable to having a tutor explain difficult concepts. This is why in my mind the real problem is that the incentives to use AI as a crutch are just too strong.
I’ve noticed you are posting a lot of studies around, some of which have been peer reviewed and some not, some argue against your point, and some show mixed results.
Are you a researcher in the pedagogical sciences? Regardless, you have to admit that the original claim has very little evidence behind it despite being testable. And also the caveat you tag onto the end is a pretty massive caveat, and from the sources you provided it seems that students which use in the way which you claim has been shown to be effective, that those students are in a minority anyway.
I think many on here are legit delusional and often talk about subjects with a lot of confidence when they clearly have a lot of mangled/broken knowledge - e.g. finance.
Yes, I agree, the skills are orthogonal. Digital typesetting is vastly quicker than manually putting down metal type, and since you’re exposed to more type you have the opportunity to learn faster. But getting good at typography with digital tools will help you very little if you need to lay out type manually.
>I think the only real case to be made is that AI's sense of immediate pleasure can neuter people away from running into friction. AI natives likely won't understand friction and question it.
This is key, I think, and gets overshadowed by people being offended by seeing bad vibecode or claims of 10x speeds, etc.
The most important learning that happens is not when we ask and get the answer to our question right away. It's when we stretch ourselves to seek out the answer, fail a few times, think deeply, then perhaps after a nap, solve the problem. That kind of knowledge is priceless because it not only gets you an answer it gets you some errant paths you can use to avoid problems in future problem solving as well as getting you increased trust in your own thinking.
If the next generations skip this step, they'll always think answers are supposed to be easy to find and will find themselves more and more dependent on AI and less and less confident in their own brains.
> If the next generations skip this step, they'll always think answers are supposed to be easy to find and will find themselves more and more dependent on AI and less and less confident in their own brains
This seems like a very polite way of saying they will become less intelligent and less capable
You’re not as senior as you think if you think reading code isn’t worth it. Do you think novelists just write novels from nothing? They read books. Software developers need to read software, too. When was the last time you read the code for the best open source software in your industry? I routinely read the libraries I use.
With anything you learn, sure, you need to read it, but you haven't actually learned it until you try to do it.
As millions of teenagers find out in high school, it is not possible to "learn" trigonometry or calculus by reading the problems; they actually have to drill problems to pass.
> Do you think novelists just write novels from nothing? They read books.
Excellent example! Even with novelists, and professional authors, they only get better by writing. Face it - millions of people read just as much as (even more) than best selling authors, and yet those millions are unable to produce anything of note.
> When was the last time you read the code for the best open source software in your industry?
All the time; how else would I know that simply reading is not sufficient to learn something?
I'm surprised that this point is even in contention; it is almost common knowledge that you can't learn from reading alone; it's the practice that results in learning, not the reading.
You only said “You don't learn by reading” which I took literally what do you expect. I’m triggered by senior engineers who don’t read code, luckily you’re not one of them. Your original comment was terse as hell.
Also, “classy.” As a phrase is about as “classy” as my original comment, so i guess we’re both passive aggressive jerks
> has the ability to LEARN so much faster with an AI research assistant, and that becoming an expert has accelerated for those with the personal stamina to dig deep (this as a requirement hasn't changed)
If anything it allows to be as lazy as possible. I have not seen anyone digging deeper with the AI tools.
I have been having a blast going back through topics I learned in college and haven't used in years. Being able to rubber duck specific questions and follow a path based on what I remember vs don't is much faster with LLM than it would be with textbook. However, I'm doing this because it is personally fun.
I'm guessing if presented with a task I wasn't interested in the LLM would create exactly the opposite outcome. Thankfully I'm at a point in my career where I don't have a lot of stuff forced on me externally so this hasn't come up, but I can picture teenage me taking a much lazier path with a much different end result.
If you decide to dig deeper, it's an incredible tool. Getting a summary of the internals of something you only use as an API, then getting it to test you on it until you understand. It really allows you to learn a lot.
> that becoming an expert has accelerated for those with the personal stamina to dig deep (this as a requirement hasn't changed)
This is a contradictory statement imo.
Digging deep still takes the same amount of time it used to. AI accelerates the surface level (badly, tbh), it doesn't accelerate digging deep. Becoming an expert still takes time and effort, there really aren't shortcuts.
To torture the Iron Man metaphor a bit. If you're not an expert without the AI, then you're not an expert with it.
Actually i think AI makes junior devs dumber. Senior devs learned over the gigantic mountains of failed projects they've built. I still get people telling me to build flat file databases - nope. Build me a micro service architecture with 50+ lambdas - nope. Been there, done that. I know why you really shouldnt if you technically can. For me AI make me go 100 mph in the right direction. I see junior devs going 100 mph toward the ocean or into a wall. "lets use ai to build a chatbot in javascript that runs on my desktop that everyone can connect to with a claude skill!" ok buddy. Are they learning? maybe when they get pwned but what I see mostly is the cambrian explosion of bad ideas. What I see is junior devs that no longer understand what a reverse proxy is because AWS made us dumber. Junior devs that can't understand memory management because higher level languages made us dumber. AI is another link in that chain. 10 years from now pretty sure most devs won't be able to read code.
My wife used to work at Acoustic Guitar magazine. She said the most common sales line to sell a guitar at Guitar Center was "it looks good on you". The sound of guitars might not be aesthetics, but in regards to sales, it most certainly is. Everyone plays the same guitars because they grew up seeing their idols play those guitars.
Yeah, I'd say guitars are heavily based on aesthetics, but not entirely. One of the reasons why I switched from a Les Paul, which was the guitar shape I loved the most before I started playing, to a Stratocaster was that the Stratocaster was just that much more comfortable. The LP body is basically just a block of wood with a cutout for your hand in comparison to the contoured body of the Stratocaster. If I had to buy an LP-style guitar again, I'd want at least a belly cut on it.
Look into a "Recording" model Les Paul (or the newer "Iridium" model if you don't care about the low-impedance electronics) - they have exactly that. The main complaint is usually the weight though - of the two I have one is 11lbs and one is 15lbs (!!!).
Counter Slayer uses the same UI library I hand built for Table Slayer[0], so it was a little easier for me to get that one going. I used to run design for Elastic, and built their React design library (with some great coworkers) so I'm used to heavy visualization work (having redesigned Kibana years ago). It's just fun for me to take all that experience and build weird little projects as far as I can take them!
I built Counter Slayer specifically for 3D board game inserts, and that's really all I use it for. It sits on the shoulder of Svelte and Threlte (Three.js) for most of the hard stuff.
Being a user of your own product is everything. Every designer I've ever hired were good, not so much because they were a great designer, but because they understood the product and could sit in the user's seat.
This is why I'm so thankful I went with Lucia early. They sort of sunset their library and replaced it with documentation (and some small utilities) for how to manage and host authentication for yourself. It's always presented as some big, scary thing you can't manage yourself, but I found that taking the week to learn how security and basic salting works, I was able to feel more confident about how everything worked.
I remember when they deprecated the library and instead made it a learning resource on implementing auth from scratch. Brilliant decision, much respect to the author.
Stores are now putting QR codes for pricing, not listing the prices out on stickers/paper. You check on your phone, and often times walk through "scan and go" making direct payment on your phone.
This is often done in stores where they say that prices can change daily, and that these tools help them keep prices up to date. The darker pattern is what this law prevents, and that even with this sort of labeling, they can't charge you different from what they charge me in the same store.
I've looked online and can't find any real examples of QR code-only stores.
It seems like QR codes are growing in popularity as a way to look up more product details, user reviews, etc. -- especially at electronics stores.
But the idea of prices being hidden entirely doesn't seem to exist anywhere in normal consumer stores. There seem to have been some store experiments and retail "concepts" (prototypes not rolled out), but it seems like the consumer backlash is extremely strong so these experiments have stopped. Consumers want to be able to browse prices, that's pretty fundamental.
There's a store in my neighborhood that only does app purchases and a little robot brings it out to your car. It's nice getting cilantro that hasn't been picked over by a hundred people, but I don't like shopping for groceries online, and AFAIK you can't actually enter the store.
I find this essay from Roger Ebert pretty wild in context of the times. I always think about it whenever I listen to the latest advances in AI voice technology.
Hey, I'm the OP. I originally started with FreeCAD. There's not much to "hook up" to Claude. It can natively write for FreeCAD. You don't need to use the FreeCAD editor and can point to an external, local file with an import. At that point there's not much more than pointing your LLM to that file. You'll need to tell the FreeCAD desktop app to update on changes.
Eventually I moved to JSCAD for the application mentioned in my blog post because I realized I wanted a more complex UI (which meant a web app) than what FreeCAD provided natively. If you're looking for something simple with some var statements though, FreeCAD might be enough.
In my experience, the MCP isn't really needed. Claude at least already can write the code pretty well. The problems are more with getting it to understand the output, which the blog post covers.
I can see it working with scad, and then having that generate some things. I'd imagine it'd struggle with an STL file. I don't know much about the format of FCStd files but I'd find that surprising if it worked fine. Obviously three.js code and everything it could be alright with.
It might be my lack of knowledge, because I've mostly just used Freecad to create and edit things and then just exported to STL (which doesn't feel like the thing Claude would be good at modifying)
Over the last year I've been hacking on Table Slayer [0] a web tool for projecting DnD maps on purpose built TV-in-table setups. Right now I'm working on making hardware that supports large format touch displays.
Since I also play boardgames, this past month I threw together Counter Slayer [1], which helps you generate STLs for box game inserts.
Both projects are open source and available on GitHub. I've had fun building software for hobbies that are mostly tactile.
What folks seem to avoid is that a Junior (in ANY subject) has the ability to LEARN so much faster with an AI research assistant, and that becoming an expert has accelerated for those with the personal stamina to dig deep (this as a requirement hasn't changed). I spend just as much time with my AI tooling asking questions as I do asking it to "build" or "fix" things. "How does this work?". "Can you suggest other tools?".
I think some people always think about AI as an input / output relationship, when a lot of the time, the fiddling in between, with or without AI was always the important part. Yes people will suck in the beginning, against they always did. I think the good folks though will suck for a MUCH shorter time than I did getting into things.
A lot of people will drop out and get discouraged. That happened before too. Learning things requires persistence. I think the only real case to be made is that AI's sense of immediate pleasure can neuter people away from running into friction. AI natives likely won't understand friction and question it.
reply