Logitech MX Ink: Target ray space not aligned with controller grip

[ferminLR]

The target ray is not correctly positioned and aligned in the logitech-mx-ink profile. The model matches the physical position and orientation of the device, and when comparing when the model drawn by the OS (by clicking in the menu button, so the OS windows appear) the models match but the rays don't. See attached images

Left: The model and its ray, properly aligned, as drawn by the OS shell.
Right: Inside WebXR (the profile viewer), the model is correctly posed, but the target ray isn't.

Some additional info:

• There is a transformation done to the controller mesh, because the target ray is just a line across the controller local Z axis, and the controller meshes axes, when viewed in Blender, don't match that direction.
• For any profile, the target ray viewed in the WebXR emulators in PC Chrome and Firefox, the Quest Browser, and Wolvic, are different; it seems the controller mesh transformation is browser dependant.
• AFAIK, There is no way to override this transformation of define the target ray space in the profile definition.
• The mesh could be edited to align the ray, of course, but that would make it not matching the device pose in real life.
• Other profiles suffer similar problems, e.g. magicleap-one

I understand this hasn't been a problem before because usually each platform only have had a small number of controllers compatible, so the browsers on those platforms had the model transformations setup for their controllers. Now that Meta is opening up the OS for third party controllers, would it make sense to add an optional definition of the targetRaySpace in the profiles?

[AdaRoseCannon]

Target Ray space is a fundamental property of the InputSource, fixing it in WebXR Input Profiles wouldn't be appropriate because some websites may choose not to use it.

I think in this particular case it should probably be fixed in Meta Quest Browser. What do you think @cabanier?

[cabanier]

I think in this particular case it should probably be fixed in Meta Quest Browser. What do you think @cabanier?

I pass the ray space from OpenXR. Maybe I should move it to the tip and align it?
I believe the spec is silent on the use case of a styles. Maybe we need to add language to the spec that says that for stylus devices, the origin of the ray is the stylus tip?

[ferminLR]

Maybe it should be fixed at an OS level? In the shell the ray space is correct, so maybe they manage the difference for this device somehow there?

Also, if you setup the stylus for left-handed use in settings, the ray points to the right of the stylus, instead of to the left like in the image attached above (the screenshot was taken in right-handed mode).

[cabanier]

/tpac how should we surface stylus type controllers?

[alcooper91]

Relevant bits of the spec that could be tweaked:

The targetRaySpace attribute is an XRSpace that has a native origin tracking the position and orientation of the preferred pointing ray of the XRInputSource (along its -Z axis), as defined by the targetRayMode. (https://immersive-web.github.io/webxr/)

"tracked-pointer indicates that the target ray originates from either a handheld device or other hand-tracking mechanism and represents that the user is using their hands or the held device for pointing. The orientation of the target ray relative to the tracked object MUST follow platform-specific ergonomics guidelines when available. In the absence of platform-specific guidance, the target ray SHOULD point in the same direction as the user’s index finger if it was outstretched."

So I think my impression was that we could just tweak the wording about "platform-specific guidance", and then provide some of that guidance.

[cabanier]

So I think my impression was that we could just tweak the wording about "platform-specific guidance", and then provide some of that guidance.

The spec talks about orientation here and not position.
Maybe we should add:

If the held device is expected to interact with the real world (such as a pen), the target ray MUST originate at that point``

[alcooper91]

I have vague recollections of discussions about something like devices that have an explicit or implied point whether they are meant to interact with the real world or not?

Could also extend to say "The position and orientation of the target ray relative to the tracked object..." and then start/formalize some place to provide controller-specific guidance.

Regardless, the suggestion sounds fine to me, but I don't recall if there was a desire for that even more generic language around devices that should have origins set as such.

[toji]

We did talk about "if the controller has a pointy bit" during the TPAC meetings, but we'd want to formalize that in a way that doesn't accidentally imply that decorative bits on a more "aggressively" styled device aren't considered the pointy part.

If the held device is expected to interact with the real world (such as a pen), the target ray MUST originate at that point

This sounds good, though I maybe think that "interact with the real world" is a bit ambiguous. Perhaps "If part of the device is intended to contact real-world surfaces (such as a pen tip)..."

However, I do wonder if that's problematic for something like the Quest Pro controllers where most of the time they act as a typical controller but can have a pen nib inserted into the bottom of the handle for writing. Would either variation of the language above imply that when the pen nib is inserted the target ray MUST start pointing out the bottom of the handle? Is that state even detectable?

[cabanier]

If the held device is expected to interact with the real world (such as a pen), the target ray MUST originate at that point

This sounds good, though I maybe think that "interact with the real world" is a bit ambiguous. Perhaps "If part of the device is intended to contact real-world surfaces (such as a pen tip)..."

OK. I'll make a PR with that language

However, I do wonder if that's problematic for something like the Quest Pro controllers where most of the time they act as a typical controller but can have a pen nib inserted into the bottom of the handle for writing. Would either variation of the language above imply that when the pen nib is inserted the target ray MUST start pointing out the bottom of the handle? Is that state even detectable?

It is not detectable. I agree it would be nice if it did.