LoginGet started

Devin will take your job soon? 500$/month for a coding assistant!

Harkirat Singh・46 minutes read

AI tool Devon, available for a $500 monthly subscription, struggles with coding tasks, showing limited performance improvements and possibly offering less value than hiring a human engineer. Despite its potential, users require technical expertise for effective use, and the current pricing is deemed excessive, prompting concerns about its long-term viability in the market.

Insights

  • AI and automation, exemplified by the AI Junior Engineer named Devon, are raising concerns about the future of coding jobs, as initial tests show Devon struggles with many coding tasks, indicating that users might find better value in hiring human engineers instead of relying on such tools.
  • The process of integrating Devon into workflows involves connecting platforms like GitHub and Slack and requires a solid understanding of coding, as the AI's performance has been inconsistent; while it can handle certain tasks like database management and code generation, it often misinterprets issues, suggesting that non-technical users may face challenges when utilizing its capabilities effectively.

Get key ideas from YouTube videos. It’s free

Recent questions

  • What is artificial intelligence?

    Artificial intelligence (AI) refers to the simulation of human intelligence processes by machines, particularly computer systems. These processes include learning, reasoning, problem-solving, perception, and language understanding. AI can be categorized into two main types: narrow AI, which is designed for specific tasks, and general AI, which aims to perform any intellectual task that a human can do. The technology is increasingly integrated into various applications, from virtual assistants and recommendation systems to autonomous vehicles and advanced robotics. As AI continues to evolve, it raises important questions about ethics, job displacement, and the future of human-machine collaboration.

  • How does automation affect jobs?

    Automation significantly impacts jobs by streamlining processes and increasing efficiency, often leading to the displacement of certain roles while creating new opportunities in others. As machines and software take over repetitive and mundane tasks, workers may find their roles evolving to focus on more complex, creative, or interpersonal aspects of work. While automation can enhance productivity and reduce costs for businesses, it also raises concerns about job security and the need for workforce reskilling. Industries such as manufacturing, retail, and even professional services are experiencing shifts due to automation, prompting discussions about the future of work and the importance of adapting to technological advancements.

  • What is a coding bootcamp?

    A coding bootcamp is an intensive, short-term training program designed to teach individuals programming and software development skills. These bootcamps typically focus on practical, hands-on learning, covering various programming languages, frameworks, and tools essential for web and software development. Programs can last from a few weeks to several months and often culminate in projects that participants can showcase to potential employers. Coding bootcamps have gained popularity as an alternative to traditional computer science degrees, providing a faster and more focused path to entering the tech industry. Graduates often pursue careers as software developers, web developers, or data analysts.

  • What is a pull request in coding?

    A pull request (PR) is a method used in version control systems, such as Git, to propose changes to a codebase. When a developer wants to merge their changes from one branch into another, they create a pull request, which allows team members to review the proposed modifications before they are integrated. This process facilitates collaboration, as it enables discussions about the changes, code reviews, and the opportunity to address any issues or improvements. Pull requests are essential for maintaining code quality and ensuring that multiple contributors can work together effectively on a project, making them a fundamental aspect of modern software development workflows.

  • What is the purpose of pagination in web design?

    Pagination in web design serves to divide content into separate pages, enhancing user navigation and improving the overall user experience. By breaking up large amounts of information into manageable sections, pagination allows users to easily browse through content without feeling overwhelmed. This is particularly important for websites with extensive data, such as e-commerce platforms or blogs, where users may need to sift through numerous items or articles. Effective pagination can also improve site performance by reducing load times, as only a portion of the content is loaded at any given time. Overall, pagination is a crucial design element that helps maintain clarity and usability in web applications.

Related videos

Summary

00:00

AI Junior Engineer Devon's Limited Coding Abilities

  • AI and automation are set to transform society, raising concerns about the future of coding jobs and the role of engineers in the workforce.
  • The AI Junior Engineer, named Devon, is now publicly available for a $500 monthly subscription, allowing users to test its capabilities in coding.
  • Initial tests revealed Devon struggled with two out of three coding issues, indicating limited improvement in its performance over the past six months.
  • Devon integrates with Slack, creating pull requests directly in codebases, but its underlying large language model (LLM) remains unchanged from previous versions.
  • Users are advised that unless there is a significant improvement in LLMs, results will be similar to those achieved with tools like GitHub Copilot.
  • The $500 subscription is considered expensive, with the reviewer suggesting that hiring a human engineer may provide better value for the same cost.
  • Onboarding for Devon involves connecting GitHub and Slack, with a straightforward process that includes authorizing access to repositories.
  • Users must install dependencies as they would for a new hire, ensuring Devon has the necessary tools to function effectively.
  • After onboarding, users can assign issues to Devon, which can verify repository access and create draft pull requests based on simple changes.
  • The reviewer notes that while Devon shows promise, it may not be suitable for non-technical users, as understanding code is essential for effective use.

15:41

AI Attempts Pagination Fix on 100exsteps.com

  • The process began with testing AI capabilities, taking approximately 5 minutes to confirm functionality and generate a simple pull request (PR) to update the README file automatically.
  • The focus shifted to identifying a real issue on the platform 100exsteps.com, specifically the need for pagination on project listings to enhance user navigation.
  • Pagination should be implemented on both the front end and back end, allowing users to navigate through multiple pages of projects seamlessly.
  • A specific PR was created to address pagination, involving around 28 lines of code changes, and the AI was tasked with generating a draft PR for this issue.
  • At 12:37 PM, the AI generated a plan to fix pagination controls, but it misunderstood the problem, focusing on navigation buttons rather than rendering issues.
  • The AI's plan included creating a new branch, adding a router hook, and updating pagination methods, but it lacked clarity on the actual problem to be solved.
  • The AI executed its plan, running a development server to test navigation, but encountered issues with authentication and database URL configurations.
  • The AI successfully created a PostgreSQL database and generated necessary files, demonstrating its ability to handle database migrations and configurations.
  • The AI's pull request included changes for client-side navigation, but it incorrectly addressed pagination in the wrong context, potentially slowing down user experience.
  • The exploration of simpler issues continued, with a focus on fixing layout problems and unresponsive images, aiming to find tasks more suitable for AI resolution.

30:53

AI Code Fixes and User Testing Process

  • The process begins with an attempt to fix a coding issue, estimated to take around 10 minutes, while also checking for reproducibility in the project steps.
  • The user tests the mobile view on an iPhone 12 Pro, noting issues like text overflow and excessive black space on the right side of the screen.
  • The user accesses the knowledge repository for Codex Flash, which generates context around the code, while waiting for the AI to process the issue.
  • The AI creates a pull request to update the package lock file and modifies the icon size, but the user doubts its effectiveness in solving the original problem.
  • The AI adjusts the height of an element to 300 pixels on smaller screens, which partially addresses the visual issue but does not fully resolve it.
  • The user fabricates a new issue regarding the profile section, where a pencil icon should allow users to edit their name, requiring both front-end and back-end changes.
  • The user estimates that the front-end changes will take 20-30 minutes, while back-end modifications should take about 5-10 minutes to implement.
  • After 10 minutes, the AI generates code for both front-end and back-end, which aligns closely with the user's expectations for functionality and structure.
  • The generated back-end code checks user sessions and updates the name in the database, while the front-end code manages the editing state and user interactions.
  • The user plans to test the AI-generated code locally to ensure it functions correctly and addresses any potential vulnerabilities before final implementation.

45:52

Tool Performance and GitHub Integration Insights

  • The author discusses the ability of a tool to push changes directly to their GitHub repository, initially expressing concern but later confirming it can indeed push to a branch.
  • To set up the project, the author recommends running `npm install` or `yarn install`, followed by `yarn dev` to ensure all dependencies are installed and the application runs correctly.
  • The author identifies an issue with the login functionality, noting that the Google login isn't working locally and that an update to the Node version is required.
  • A problem arises where changes made to the user's name do not reflect immediately due to the name being stored in the session, necessitating a logout and relogin to see updates.
  • The author suggests that user details should be fetched from the database instead of being stored in the session, emphasizing that only the user ID should be kept in the session for efficiency.
  • The author critiques the tool's performance, stating it is decent but prefers using a local language model (LLM) over cloud-based solutions, citing concerns about accuracy and pricing.
  • Final thoughts indicate that while the tool has potential, its current pricing at $500 per month is seen as excessive, with expectations for future price reductions as the market evolves.
Channel avatarChannel avatarChannel avatarChannel avatarChannel avatar

Try it yourself — It’s free.