Before performing the study, the developers in question expected the AI tools would lead to a 24 percent reduction in the time needed for their assigned tasks. Even after completing those tasks, the developers believed that the AI tools had made them 20 percent faster, on average. In reality, though, the AI-aided tasks ended up being completed 19 percent slower than those completed without AI tools.

  • usernamesAreTricky@lemmy.ml
    link
    fedilink
    arrow-up
    6
    ·
    6 days ago

    Though also when breaking down the study earlier to more experience developers, a similar same pattern of within margin of error change or decrease in productivity shows in that metastudy

    They are also not comparing the same metrics here. The earlier study is looking at number of commits and pull requests as a metric for productivity. The other is looking at the time per task

    Number of commits / PRs / similar kinds of metrics like lines of aren’t great for measuring productivity in general and especially here. Usage patterns with AI could very easily change your commit pattern and PR patterns without changing how much you are getting done