this post was submitted on 01 Oct 2024
73 points (100.0% liked)

Technology

20 readers
8 users here now

Which posts fit here?

Anything that is at least tangentially connected to the technology, social media platforms, informational technologies and tech policy.


Rules

1. English onlyTitle and associated content has to be in English.
2. Use original linkPost URL should be the original link to the article (even if paywalled) and archived copies left in the body. It allows avoiding duplicate posts when cross-posting.
3. Respectful communicationAll communication has to be respectful of differing opinions, viewpoints, and experiences.
4. InclusivityEveryone is welcome here regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, education, socio-economic status, nationality, personal appearance, race, caste, color, religion, or sexual identity and orientation.
5. Ad hominem attacksAny kind of personal attacks are expressly forbidden. If you can't argue your position without attacking a person's character, you already lost the argument.
6. Off-topic tangentsStay on topic. Keep it relevant.
7. Instance rules may applyIf something is not covered by community rules, but are against lemmy.zip instance rules, they will be enforced.


Companion communities

!globalnews@lemmy.zip
!interestingshare@lemmy.zip


Icon attribution | Banner attribution

founded 1 year ago
MODERATORS
 

The study tracked around 800 developers, comparing their output with and without GitHub's Copilot coding assistant over three-month periods. Surprisingly, when measuring key metrics like pull request cycle time and throughput, Uplevel found no meaningful improvements for those using Copilot.

top 10 comments
sorted by: hot top controversial new old
[–] AnarchoSnowPlow@midwest.social 11 points 1 month ago (2 children)

I've tried it for even some boiler plate code a few times. I've had to end up rewriting it every time.

It makes mistakes like Junior engineers, but it doesn't make them in the same way that junior engineers do, which means that as a senior engineer it takes me significantly more effort to review. It also makes mistakes that humans don't, which is even weirder to catch in review.

[–] leisesprecher@feddit.org 5 points 1 month ago

Also my experience. It sometimes tries to be smart and gets everything wrong.

I think code shows clearly, that LLMs don't actually understand what's written. Often enough you can clearly see it trying to insert a common pattern even though that doesn't make sense at this point.

[–] TrippaSnippa@lemm.ee 2 points 1 month ago

As a junior-to-mid-level developer I find myself having to rewrite the boilerplate code copilot comes up with as often as not, or it will get things slightly wrong that I then have to go back and fix. I'm starting to think that most of the time it would be just as quick for me to just write it all myself.

[–] rushaction@programming.dev 10 points 1 month ago

It's a glorified autocorrect. Using it for anything else and expecting magic is an interesting idea. I'm not sure what folks are expecting there.

  • It suggests variables and function names I was already going to type more accurately. Better, it suggests ones even when I cannot remember the name because i got stumped trying to remember.
  • It fills in basic control structures and functions more effectively than the IDE's completion templates.
  • It provides a decent starting point for both comments and documentation (the hard part). Meaning my code actually has comments and documentation. Regularly!

But I don't ask it to explain things or generate algorithms willy nilly. I don't expect or try to have it do something that's not more than simply auto-completion.

I honestly like it, even if I strongly dislike the use of AI elsewhere. It's working in this area for me.

[–] stoy@lemmy.zip 6 points 1 month ago

The few times I have used AI to help me with coding has mostly been to ask it for examples on how to use a specific feature, then it has been ok for the most part.

I mostly code in PowerShell, HTML and CSS, and Bing Chat helpful when I am stuck on a small issue.

We also recently started testing Copilot Pro 365, the one that can help you make documents or search through company documents and stuff like that.

As a test I asked it to make me a powerpoint presentation about the top ten podcasting microphones to buy.

The result looked great at first glance, but quicly got very generic.

Sure, it did show pictures of some microphones and even spoke about them, but it was just vauge and generic

[–] LiveLM@lemmy.zip 5 points 1 month ago* (last edited 1 month ago)

I got a bridge to sell to anyone who thought AI would help reduce burnout lmao

No really, AI has great uses but I'm in awe anyone thought this was one of them

[–] floofloof@lemmy.ca 4 points 1 month ago

I use it sometimes to ask whether there's an API for whatever I need to do. But it lies.

[–] Sl00k@programming.dev 4 points 1 month ago

Surprisingly, when measuring key metrics like pull request cycle time and throughput, Uplevel found no meaningful improvements for those using Copilot.

This assumes I'm going to dedicate my increased productivity to my employer. I'm still at the same level of productivity but personally my effort needed for specific tasks has dropped a lot meaning more free time for me.

[–] laurelraven@lemmy.zip 3 points 1 month ago

I'm not sure why that's so surprising actually

[–] NigelFrobisher@aussie.zone 2 points 1 month ago

It’s slightly better Resharper, except when it fucks up and then it’s just an annoying parlour trick.