• 2 Posts
  • 4 Comments
Joined 1 year ago
cake
Cake day: June 30th, 2023

help-circle

  • I’m actually working on this problem right now for my master’s capstone project. I’m almost done with it; I can have it generating a series of steps to try and fetch me something based on simple objectives like “I’m thirsty”, and then in simulation fetching me a drink or looking through rooms that might have a fix, like contextually knowing the kitchen is a great spot to check.

    There’s also a lot of research into using the latest advancements in reasoning and contextual awareness via LLMs to work towards better more complicated embodied AI. I wrote a blog post about a lot of the big advancements here.

    Outside of this I’ve also worked at various robotics startups for the past five years, though primarily in writing data pipelines and control systems for fleets of them. So with that experience in mind, I’d say we are many years out from this being in a reasonable product, but maybe not ten years away. Maybe.





  • I just finished my first long technical blog post - blogging is a relatively new habit I’m trying to build up. I posted the blogpost already to the robotics community so I’ll just crosspost it with this link. TLDR it’s the trial and tribulations I had while trying to use PPO to train a robotic arm to do pick and place.

    I’m putting a hold on the other project I was burning out of - something I called coppermind (yes I’m a Sanderson fan) that handled chatbot memory/applications. I have an old version running on a digital ocean droplet plugged into Twilio, but recent degradations in ChatGPT 3.5 model response quality + that version not having my knowledge/vector database features means it’s becoming a bit repetitive on its responses.

    Right now with my masters in robotics starting up again in late August, and this being the project for the entire thing, I’m spending some time working on tooling to get myself up to speed on ROS2 and get repeatable environments up and running for it so I can quickly fix/deploy sim robots for the project. I have some ideas I’m toying with there. Unfortunately no idea what the project will be yet.