gs.generate() API and documentation

[volezheng]

Hi, the gs.generate() demo is really cool! However, I did not find it in the user guide and API Reference. Appreciate if there is some pointer. Thanks!

[TomatoPicker278]

yes! I also try it, but i can't find the generate() function.

[max-titov]

from the main page: "Currently, we are open-sourcing the underlying physics engine and the simulation platform. Access to the generative framework will be rolled out gradually in the near future."

[jacobprice808]

Damn, that's disappointing. Would have been an amazing contribution to the open source community...

[bonillam]

try setting up tool calling with a system prompt containing few shot examples using:

• pydantic class definitions for the genesis object
• scene text description
• the API tool calling

worth a try experimenting with that...maybe

[JocelynVelarde]

Yea I was trying to test it and got this.
I was very excited to test this out.

[ti5robot]

i didn't find generate function,please help me

[JocelynVelarde]

ahhh because it does not exist.

[raspiduino]

I came here for this, but also left with disappointment :(

[younes-io]

I spent hours to get it to work on WSL2 (which it doesn't) then tried the demo code and it failed !!! Really disappointing to "sell" something that doesn't exist yet..

[zhouxian]

Thank you for your interest ! But as our read me clearly indicates, that part requires a quite a bit of extra work before it’s ready for public use, so stay tuned

[younes-io]

@zhouxian I do understand it takes time, but my comment is about the fact of not clarifying this explicitly; the README states this on "What is Genesis?":

[zhouxian]

Ah I see. That’s for the heads up. Will update to make it more clear

[zhouxian]

OK we used to have a paragraph mentioning this, but somehow it got deleted during merging prs related to README. I have added it back

[CalebDeLeeuwMisfits]

Is there any updates on when/if the generative portion will be released or an API made available?

[younes-io]

I really hope to get a concrete feedback from the devs on this, I've been waiting for a while now.. I asked the question on Discord, but no feedback so far

[Genesis-Embodied-AI]

The team is taking a semi break this month and we will be back with full thrust to resume developing genesis with an engineering team in Feb, together with a concrete timeline and roadmap

Please understand - we are a bunch of students before we form our engineering team next month

Regarding the status of gs.generate: there are two major reasons we are not releasing it now: 1. the demo we showed in release uses Genesis's ray tracer rendering, which is photorealistic but very slow (although already a few times faster than other available raytracer on the marke). To make generative function more usable, we will re-build a complete rendering pipeline combining rasterization and raytracing, to achieve similar photorealistic rendering quality while giving real-time rendering speed 2. the modules we have right now are routed using a high-level LLM agent which invokes each generative modules separately. We want to unify these and be able to generate complex worlds that integrates different modules, e.g. character motion and robot motion all together. This requires quite a bit of extra engineering work

[younes-io]

Thanks for the feedback. Please keep in mind that communication is very important =)
I wish you the best for the next steps, and can't wait to see the outcome of your engineering team's work! Break a leg!

[kevenzhou]

disappointed too

[Genesis-Embodied-AI]

see above