Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Port mouse to Android Phones? #3

Closed
virresh opened this issue Mar 6, 2021 · 49 comments
Closed

Port mouse to Android Phones? #3

virresh opened this issue Mar 6, 2021 · 49 comments

Comments

@virresh
Copy link
Owner

virresh commented Mar 6, 2021

Requested by Samuel ([email protected]) over email for Android phones without a touchscreen

Example model -- https://www.lg.com/us/cell-phones/lg-VN220-verizon-exalt#

Expectation -- Mouse enabled via VoiceCommand button [key 60 TV_INPUT_HDMI_1]

Since these models look like they're dumbed down version Android 6, I'm not too hopeful of them supporting most APIs
Also since it's Android 6, it'll not have support for gesture API, so we're stuck with Android getting things right via the performAction API (which IMO is highly unreliable)
@virresh
Copy link
Owner Author

virresh commented Mar 6, 2021

https://github.com/virresh/matvt/releases/download/phone-v1.01/matvt-phone-app-release-v1.01.apk

Whoever is interested, here is a simple port for phones
Please use it by overriding key to whatever you wish to, default is mute key

Should work best with Android N onwards, but have added support for legacy actions (please note that these are totally unreliable and might not even work in many cases) for Android M.

@lgexalter
Copy link

I have tested this new phone version on the LG Exalt but there is a problem. The UI of these phones is severely dumbed down and the only way to grant permissions is through ADB due to the option being missing from the settings. I have granted the overlay permission through adb successfully. When granting accessibility to the app the process which has worked for other accessibility apps (e.g. Voice Access) is to set the app as enabled_accessibility_services (with adb shell settings put secure enabled_accessibility_services package.name.here/service.name.here) that works no problem. But when trying to start the accessibility service (adb shell am startservice package.name.here/service.name.here) it throws an error Error: Requires permission android.permission.BIND_ACCESSIBILITY_SERVICE. And android.permission.BIND_ACCESSIBILITY_SERVICE cannot be granted with adb. The app therefore remains on the first screen and says Accessibility Permissions: Not Granted Accessibility Services: Not Running.

@virresh
Copy link
Owner Author

virresh commented Mar 8, 2021

I haven't seen this error before. My best guess would be that the drawing over other screen permission is missing.
Can you confirm if the draw over other apps permission was granted?
Also probably you're trying the same thing, but just to confirm you've already tried the steps at https://android.stackexchange.com/a/231062 and are getting an error post that?

@lgexalter
Copy link

Yes I have granted the drawing over other apps permission successfully.
And I've also followed the steps from https://android.stackexchange.com/a/231062

@virresh
Copy link
Owner Author

virresh commented Mar 12, 2021

@lgexalter unfortunately we're out of luck here.

I don't think I can do much without being able to actually see what's happening. Sorry mate.

Leaving this issue open, just in case some developer who owns a similar device and happens to be interested in this comes across this thread.

@LGExalt
Copy link

LGExalt commented Mar 23, 2021 •
edited
Loading

@lgexalter what were the exact commands that you used to;
1 grant accessibility services?
2 to start the accessibility services?
3 to start the overlay services?

thanks

@Bo-Exalt
Copy link

@LGExalt What Commands did you do to get it working?

@Bo-Exalt
Copy link

I granted overlay but when I start accessibility it says: "Requires permission android.permission.BIND_ACCESSIBILITY_SERVICE"??

@XP3R
Copy link

XP3R commented Mar 25, 2021

I tried the app on the Sonim Xp3, which is easy to allow accesibility services without adb commands, it does not have a mute button, but I was able to change that with the override setting. However I had 2 issues, one minor one that the cursor size was pretty big no matter ther size setting, but the main issue is that the select button does not click on anything, I had this problem before with other cursors, apparently the select button on the XP3 Is different than most phones. The key code is 66 and the scan code is 352. If someone can modify the code to make that select hopefully would work

@virresh
Copy link
Owner Author

virresh commented Mar 27, 2021

I successfully installed it on my exalt ;)

@LGExalt can you elaborate a bit on the steps to install?
IMO it'd be quite helpful for a lot of folks here
I personally don't own any such phone so it'd be great if someone can chip in to fill the gap

@virresh
Copy link
Owner Author

virresh commented Mar 27, 2021

the cursor size was pretty big no matter ther size setting
The key code is 66 and the scan code is 352.

Thanks @XP3R for reporting, I'll update the app for phone with these modifications and update here.
@kfmdmteam has reported the ideal size for their phone is 20 x 20 pixels on community chat [1].
I'll adjust the cursor around these dimensions to suit your use-case

[1] https://gitter.im/virresh/community?at=605cdeab68921b62f4874241

@virresh
Copy link
Owner Author

virresh commented Mar 27, 2021

Since exact commands are not shared here, I request all the folks here to please try out the following:

adb shell appops set io.github.virresh.matvt SYSTEM_ALERT_WINDOW allow
adb shell settings put secure enabled_accessibility_services io.github.virresh.matvt/.services.MouseEventService
adb shell settings put secure accessibility_enabled 1

and let me know if these work?

@XP3R
Copy link

XP3R commented Mar 30, 2021

the cursor size was pretty big no matter ther size setting
The key code is 66 and the scan code is 352.

Thanks @XP3R for reporting, I'll update the app for phone with these modifications and update here.
@kfmdmteam has reported the ideal size for their phone is 20 x 20 pixels on community chat [1].
I'll adjust the cursor around these dimensions to suit your use-case

[1] https://gitter.im/virresh/community?at=605cdeab68921b62f4874241

Thanks! Where can I find the updated version? (I guess I can adjust the mouse size like you posted here https://gitter.im/virresh/community?at=605cdeab68921b62f4874241 but I still would have the select button not working)

@virresh
Copy link
Owner Author

virresh commented Mar 31, 2021

@XP3R I've not had the chance to update the app
I'll do it over the weekend and ping here when it's available
Hope that works

@greatjack1
Copy link

#9 Solves these issues and makes the app fully usable on LG Phones

@virresh
Copy link
Owner Author

virresh commented Mar 31, 2021

Thanks a lot @greatjack1 for chipping in!

I'll do a release over the weekend along with your contributions. Thanks!

@LGExalt
Copy link

LGExalt commented Mar 31, 2021 •
edited
Loading

Thanks to all those who contributed to make this happen.
I'd just like to raise 2 points.
1 the cursor doesn't work in some apps. For example Waze and Uber. At first I thought it's because those apps that I was using already had a cursor added. So I tried it on a different version of Waze, which didn't have an added cursor, and it still wouldn't click on anything.

2 Once I ran the adb commands, the dedicated key to turn on voice access didn't work anymore. This may be an issue with the phone (LG Exalt). Because I had a similar issue when using button mapper. Once a enabled button mapper, I couldn't set a key for voice access.
If anyone knows of a way of fixing these issues, I'd really appreciate it.
Thanks

@XP3R
Copy link

XP3R commented Mar 31, 2021 •
edited
Loading

Thanks to all those who contributed to make this happen.
I'd just like to raise 2 points.
1 the cursor doesn't work in some apps. For example Waze and Uber. At first I thought it's because those apps that I was using already had a cursor added. So I tried it on a different version of Waze, which didn't have an added cursor, and it still wouldn't click on anything.

2 Once I ran the adb commands, the dedicated key to turn on voice access didn't work anymore. This may be an issue with the phone (LG Exalt). Because I had a similar issue when using button mapper. Once a enabled button mapper, I couldn't set a key for voice access.
If anyone knows of a way of fixing these issues, I'd really appreciate it.
Thanks

When enabling services with adb you need to put in all services you want enabled at once, so typing adb shell settings put secure enabled_accessibility_services io.github.virresh.matvt/.services.MouseEventService would turn off voice access and button mapper. You need to enable them at once separated by a colon, such as pkg1/service1:pkg2/service2

@LGExalt
Copy link

LGExalt commented Mar 31, 2021 •
edited
Loading

Thanks to all those who contributed to make this happen.
I'd just like to raise 2 points.
1 the cursor doesn't work in some apps. For example Waze and Uber. At first I thought it's because those apps that I was using already had a cursor added. So I tried it on a different version of Waze, which didn't have an added cursor, and it still wouldn't click on anything.
2 Once I ran the adb commands, the dedicated key to turn on voice access didn't work anymore. This may be an issue with the phone (LG Exalt). Because I had a similar issue when using button mapper. Once a enabled button mapper, I couldn't set a key for voice access.
If anyone knows of a way of fixing these issues, I'd really appreciate it.
Thanks

When enabling services with adb you need to put in all services you want enabled at once, so typing adb shell settings put secure enabled_accessibility_services io.github.virresh.matvt/.services.MouseEventService would turn off voice access and button mapper. You need to enable them at once separated by a colon, such as pkg1/service1:pkg2/service2

I've done that. And the voice access app works. I'm talking about setting a dedicated key to start it. For example the camera key.

@kfmdmteam
Copy link

kfmdmteam commented Apr 2, 2021 •
edited
Loading

It won't work because matvt grabs all accessibility keys already. So you can't double select in accessibility what a key should do. Supposedly by the LG classic you can. But I have not tested it.

@LGExalt
Copy link

LGExalt commented Apr 2, 2021

It won't work because matvt grabs all accessibility keys already. So you can't double select in accessibility what a key should do. Supposedly by the LG classic you can. But I have not tested it.

I tested it on the classic and it worked perfectly, and I was still able to set a voice access key. Not sure what the difference is. They seem to be based on the same OS.

@virresh
Copy link
Owner Author

virresh commented Apr 2, 2021

Once I ran the adb commands, the dedicated key to turn on voice access didn't work anymore.

Hmm,
Could be OS not accepting re-throws of un-used keys, but I cannot say with certainity.

matvt returns any key strokes back to the system if it cannot find a use for it.

Here's another pre-release snapshot version:
https://github.com/virresh/matvt/releases/download/phone-v1.01/matvt-phone-app-release-v1.01-snapshot.apk

@XP3R @LGExalt @kfmdmteam Can you folks please test it on the various flip phones that you have for sanity?
From kfmdmteam's comment, it probably will need more modifications on LG Exalt, but I'd still be interested to know what error is thrown by adb if we don't do those modifications

This version has some rudimentary support for long touch (should work on Android 6, doesn't trigger on Android 7+)
It has a more customizable size range, so should not appear too big and have added the extra dpad down button presses, so clicking should now work

@kfmdmteam
Copy link

The vn220 is android 6 while the lg classic is android 8

@kfmdmteam
Copy link

kfmdmteam commented Apr 2, 2021 •
edited
Loading

@virresh I see this version doesn't have the options like the v1.03 would it be possible to make the phone version built in the newer one?

@virresh
Copy link
Owner Author

virresh commented Apr 2, 2021

Yep, I'm working on backporting them. Asking for some early feedback so that I can incorporate more changes if required with the next major release itself :D

@kfmdmteam
Copy link

Looking great!

@virresh
Copy link
Owner Author

virresh commented Apr 2, 2021

Did it work without package manipulation on the exalt phone?

@kfmdmteam
Copy link

kfmdmteam commented Apr 2, 2021 via email

@virresh
Copy link
Owner Author

virresh commented Apr 2, 2021

Ah okay. Thanks for the heads up.
Can you try with this version: https://github.com/virresh/matvt/releases/download/phone-v1.01/matvt-phone-app-release-v1.01-snapshot-2.apk

It has the required package manipulation

adb shell appops set com.android.cts.virresh.matvt SYSTEM_ALERT_WINDOW allow
adb shell settings put secure accessibility_enabled 1
adb shell settings put secure enabled_accessibility_services com.android.cts.virresh.matvt/com.android.cts.virresh.matvt.services.MouseEventService
adb shell am startservice com.android.cts.virresh.matvt/com.android.cts.virresh.matvt.services.MouseEventService

@kfmdmteam
Copy link

kfmdmteam commented Apr 2, 2021 via email

@kfmdmteam
Copy link

kfmdmteam commented Apr 2, 2021 via email

@virresh
Copy link
Owner Author

virresh commented Apr 2, 2021 •
edited
Loading

Okay I'll try but can you set the DPI for the wording in the GUI for 10 DPI

sure

Seems like there's an issue the package name is still not Android CTS

I've put the service in the cts package. Is there any special signing requirement also?
Did some more changes in packages, alongside dpi
https://github.com/virresh/matvt/releases/download/phone-v1.01/matvt-phone-app-release-v1.01-snapshot-3.apk

Thanks a lot for bearing with the testing

Service FQN: com.android.cts.io.github.virresh.matvt.services.MouseEventService

@kfmdmteam
Copy link

kfmdmteam commented Apr 2, 2021 via email

@virresh
Copy link
Owner Author

virresh commented Apr 2, 2021

Sorry about the confusion
Android Studio caching caused splits in the package
https://github.com/virresh/matvt/releases/download/phone-v1.01/matvt-phone-app-release-v1.01-snapshot-4.apk

Everything (including changes in manifest) are there in the above

@kfmdmteam
Copy link

kfmdmteam commented Apr 2, 2021 via email

@LGExalt
Copy link

LGExalt commented Apr 2, 2021

Sorry about the confusion
Android Studio caching caused splits in the package
https://github.com/virresh/matvt/releases/download/phone-v1.01/matvt-phone-app-release-v1.01-snapshot-4.apk

Everything (including changes in manifest) are there in the above

I got these errors.
Users\Home\Desktop\All Desktop\platform-tools>adb shell appops set com.android.cts.virresh.matvt SYSTEM_ALERT_WINDOW allow
Error: No UID for com.android.cts.virresh.matvt in user 0

C:\Users\Home\Desktop\All Desktop\platform-tools>adb shell am startservice com.android.cts.virresh.matvt/com.android.cts.virresh.matvt.services.MouseEventService
Starting service: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] cmp=com.android.cts.virresh.matvt/.services.MouseEventService }
Error: Not found; no service started.

@emaillk016
Copy link

it's
adb shell appops set com.android.cts.io.github.virresh.matvt SYSTEM_ALERT_WINDOW allow
adb shell settings put secure accessibility_enabled 1
adb shell settings put secure enabled_accessibility_services com.android.cts.io.github.virresh.matvt/com.android.cts.io.github.virresh.matvt.services.MouseEventService
adb shell am startservice com.android.cts.io.github.virresh.matvt/com.android.cts.io.github.virresh.matvt.services.MouseEventService

@XP3R
Copy link

XP3R commented Apr 2, 2021 •
edited
Loading

I'm happy to report that the snapshot works great on the Sonim XP3, it even works on waze [some people reported it not working on waze on other phones]. The only issue I see is that in scroll mode it scrolls all the way to the bottom\top with each click. I didnt see any difference in performance between snapshot one and snapshot 4, except snapshot one had all the settings in the accessability section in addition to on the app itself, but either way is fine. It also worked with other dedicated keys for switchboard and voice access. Long pressing clicks does not seem to work though, the XP3 runs android 8.1, it would be nice if it worked, but not especially neccasary. Thanks so much!

@Bo-Exalt
Copy link

Bo-Exalt commented Apr 2, 2021

The mouse for exalt still needs modding. waze not working.

@kfmdmteam
Copy link

kfmdmteam commented Apr 2, 2021 via email •
edited
Loading

@virresh
Copy link
Owner Author

virresh commented Apr 4, 2021

Thanks everyone for the feedback

The mouse for exalt still needs modding

@Bo-Exalt can you explain a bit what kind of modding was required to make it work with the app you mentioned?

I got these errors.

Thanks @LGExalt for the errors.
Do you still get the errors after trying commands posted by @emaillk016 ?

@Bo-Exalt
Copy link

Bo-Exalt commented Apr 5, 2021

It installs perfectly fine, just for a lot of things it doesn't click i.e. waze

@LGExalt
Copy link

LGExalt commented Apr 6, 2021 •
edited
Loading

Thanks @LGExalt for the errors.

Do you still get the errors after trying commands posted by @emaillk016 ?
@virresh The commands seem to have worked. And when I open the app it says that all permissions are granted and everything is started. However I can't figure out how to enable the mouse. Pressing *** Doesn't turn it on.
(My voice access key is gone though)

@4388450
Copy link

4388450 commented Apr 7, 2021

The only issue I see is that in scroll mode it scrolls all the way to the bottom\top with each click.

@virresh
Copy link
Owner Author

virresh commented Apr 8, 2021

It installs perfectly fine, just for a lot of things it doesn't click i.e. waze

Ah I see. I presume this to be happening on Android 6 devices. I'm a bit limited on Android 6 without root access, will see if something can be done about it.

I can't figure out how to enable the mouse.

There's a video made by Tech Doctor UK (https://youtu.be/UC7bPw2tG4c) that explains how MATVT works on a TV. It's the same principle, just ported to phone. You can customize the key based from settings GUI based on whatever works best for your device. Let me know if you still face issues enabling it.

scroll mode it scrolls all the way to the bottom\top with each click

Thanks, I presume this only happening on Android 7+ ?
I have a fix in mind for this issue, just didn't get enough time to address it yet. The latest version for Android TV has a scroll speed control, which should help with this problem. I'm hoping I'll get to port it to phones this weekend 🤞🏻 .

@XP3R
Copy link

XP3R commented Apr 8, 2021

scroll mode it scrolls all the way to the bottom\top with each click

Thanks, I presume this only happening on Android 7+ ?
I have a fix in mind for this issue, just didn't get enough time to address it yet. The latest version for Android TV has a scroll speed control, which should help with this problem. I'm hoping I'll get to port it to phones this weekend 🤞🏻 .

I should point out its not automatically all the way to the bottom, it just scrolls A LOT but if it's a really long page it won't go all the way to the bottom. Slowing down the scroll speed a lot should help. Thanks a million!

@LGExalt
Copy link

LGExalt commented Apr 8, 2021

Just realized that the activation key is set by default to holding down the star key.
I still don't have my voice access key though ☹️
And it won't click on Waze.

@virresh
Copy link
Owner Author

virresh commented Apr 10, 2021 •
edited
Loading

I've completed a port to Flipphones
The latest version is available at https://github.com/virresh/matvt/releases/tag/phone-v1.03
Apk can be downloaded from assets at the bottom of release notes.

Please test

Thanks to everyone who made this happen!
Cheers

@virresh
Copy link
Owner Author

virresh commented Jun 20, 2021

Keeping in mind the above comment, I've opened seperate issues for the bugs reported as they are common for the TV and phone version. Closing this issue. Feel free to comment / re-open if required.

@virresh virresh closed this as completed Jun 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
9 participants
@virresh @greatjack1 @4388450 @emaillk016 @lgexalter @LGExalt @kfmdmteam @Bo-Exalt @XP3R