MacOs ARM compile-build problems #79
Comments
|
Hi @WhiteSte I’ll update these scripts, thanks for noticing it! Usually there is no need in cross compilation for local needs, and if that’s the case just run |
|
I used to use these scripts and publish PDAL from the laptop, but since releases moved to CI it got simplified in some sense. |
|
is there something like a Running sbt publishLocal goes into o] delivering ivy file to /Users/steve/Downloads/java-main/core/target/scala-3.3.1/ivy-HEAD+20240119-1403.xml
[info] published pdal_3 to /Users/steve/.ivy2/local/io.pdal/pdal_3/HEAD+20240119-1403/poms/pdal_3.pom
[info] published pdal_3 to /Users/steve/.ivy2/local/io.pdal/pdal_3/HEAD+20240119-1403/jars/pdal_3.jar
[info] published pdal_3 to /Users/steve/.ivy2/local/io.pdal/pdal_3/HEAD+20240119-1403/srcs/pdal_3-sources.jar
[info] published pdal_3 to /Users/steve/.ivy2/local/io.pdal/pdal_3/HEAD+20240119-1403/docs/pdal_3-javadoc.jar
[info] published ivy to /Users/steve/.ivy2/local/io.pdal/pdal_3/HEAD+20240119-1403/ivys/ivy.xml
[error] java.lang.RuntimeException: Conflicting cross-version suffixes in: com.github.sbt:sbt-jni-core
[error] at scala.sys.package$.error(package.scala:30)
[error] at sbt.librarymanagement.ConflictWarning$.processCrossVersioned(ConflictWarning.scala:39)
[error] at sbt.librarymanagement.ConflictWarning$.apply(ConflictWarning.scala:19)
[error] at sbt.Classpaths$.$anonfun$ivyBaseSettings$71(Defaults.scala:3294)
[error] at scala.Function1.$anonfun$compose$1(Function1.scala:49)
[error] at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:63)
[error] at sbt.std.Transform$$anon$4.work(Transform.scala:69)
[error] at sbt.Execute.$anonfun$submit$2(Execute.scala:283)
[error] at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:24)
[error] at sbt.Execute.work(Execute.scala:292)
[error] at sbt.Execute.$anonfun$submit$1(Execute.scala:283)
[error] at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
[error] at sbt.CompletionService$$anon$2.call(CompletionService.scala:65)
[error] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error] at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
[error] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
[error] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[error] at java.base/java.lang.Thread.run(Thread.java:833)
[error] (core-scala / update) Conflicting cross-version suffixes in: com.github.sbt:sbt-jni-core
[error] Total time: 4 s, completed 19 gen 2024, 14:03:53
|
|
@WhiteSte what commands you're experiencing troubles with exactly? The intended way to publishLocal it is via $ sbt +publishLocal
My output: $ conda-activate pdal-java
(pdal-java) [pdal-java (upd/plugins)]$ sbt +publishLocal
[info] welcome to sbt 1.9.8 (Azul Systems, Inc. Java 11.0.17)
....
[info] loading settings for project root from build.sbt ...
[info] set current project to pdal-jni (in build file:/Users/.../pdal-java/)
[info] Setting Scala version to 2.12.18 on 3 projects.
[info] Excluded 1 projects, run ++ 2.12.18 -v for more details.
[info] Reapplying settings...
[info] set current project to pdal-jni (in build file:/Users/.../pdal-java/)
[info] Wrote /Users/.../pdal-java/core/target/scala-2.12/pdal_2.12-2.6.0+6-6f2649e7-SNAPSHOT.pom
[info] Wrote /Users/.../pdal-java/core-scala/target/scala-2.12/pdal-scala_2.12-2.6.0+6-6f2649e7-SNAPSHOT.pom
[info] Wrote /Users/.../pdal-java/native/target/pdal-native-arm64-darwin-2.6.0+6-6f2649e7-SNAPSHOT.pom
[info] Building library with native build tool CMake
[info] Using CMake version 3.28.1
...
[success] Total time: 3 s, completed Jan 19, 2024, 10:01:30 AM |
|
If you want to publish locally native binaries only: Also more help around SBT can be found in their docs, and questions about the sbt-jni plugin feel free to ask here / or under the https://github.com/sbt/sbt-jni repo (I'm maintainer there as well) |
|
I could make it work using What i did is to switch inside /.m2/io/pdal/pdal-native/2.6.0 the pdal-native.jar published by you with the one i have just built.. Actually i did the same the last year to make it work the version 2.4.3 (also there arm was not supported) Going from v2.4.3 to v2.6.2 is going to be a pain for me Thats the fatal error |
|
ps: I'm doing everything in java 17 . Shouldn't be a problem i hope Furthermore, my installed version is pdal v2.6.2, the only one available in nix after 2.6.0 btw, my log when building the native is quite different and contains few errors, even if it finishes |
|
@WhiteSte what is the |
|
the sbt core.test result is the same fatal error |
|
Gotcha, I had the same behavior with the "incorrectly" installed PDAL. What works for me now is PDAL installed via conda, and build (+ test) with the active conda env. What's your PDAL installation? |
|
my pdal installation is done through nix env, I've installed pdal 2.6.2. |
|
I've created a separate env in coda and installed pdal in there with Now i've built the natives and executed the Then i took the created jar and substituted into the maven directories, i ran my java app and that's the result, where at least it is saying something about C |
|
Big Update: |
|
There was a bug with |
I've took a look on it. I could run It crashes But if i comment out the filter from the pipeline it works. My deduction is that the bindings are not reflecting the new changes in the native pdal version? But maybe we need to open another issue for that, for what regard the arm java build it has been solved using conda. @pomadchin Should we update the documentation regard that? There could be a section in the readme dedicated to the macos -arm build |
|
@WhiteSte does the same pipeline work from the command line (the entire json passed as a pipeline input)? Yea we can add a section if its useful, but it's not really a part of the repo. There are lots of ways to install PDAL (: |
I can confirm , if i execute the same pipeline from cli it works. I put here the json for reference
At this point i'm quite sure that the filter is broken in the java bindings |
|
@WhiteSte could you share the .las file to debug / confirm its failing for me as well? Yes totally could be bindings / bindings build issue. |
|
I can't give you the original file as the size is 900MB I can give you this sub file that is 300kb, on which is crashing too I'm asking myself if it crashes on every file |
|
@WhiteSte what is exactly the code with the pipeline you're trying and is failing? I tried with val pipeline = Pipeline(json)
pipeline.execute()
println(s"pipeline.getMetadata(): ${pipeline.getMetadata()}")
pipeline.close()and it prints metadata successfully; could be the local env / code specific issue |
|
you can try this file, just open it and execute. Remember to change te path to test.las with your path If you comment out It works, otherwise not |
|
Ok, I was able to reproduce; crashes only with log level >= 3! I'll see what's going on there, could be a non existent log level anymore. Try setting it to 0, 1, 2 https://github.com/PDAL/java/blob/main/core/src/main/scala/io/pdal/LogLevel.scala#L28-L30 |
|
That works! How did you found out that? |
|
Just played around with it for a bit; It looks like it's a PDAL The best is to have a reproducible example in C++ without any jni / py trickery around + a ticket in the PDAL repo. |
|
how did we assumed that is a c++ pdal issue? I suppose is a jni related problem |
|
I did not assume it's a C++ PDAL issue, I assumed that if it's reproducible in bare PDAL than it's not a bindings issue. Could be totally a bindings issue (: |
|
But TBH I have hard times linking the log level setting affecting filter execution may be a bindings issue. Unless some pointer get lost somehow 🤔 |
|
Will make a PR shortly, that's indeed a bindings issue. Logger output stream is not properly initialized (?) and it's basically an NPE (??) 100% of times. It refuses to write into The only reason it never crashed before is that we were lucky enough to not have any logging statements for the specified levels (?) It works with stdout / stderr / files. |
That's strange because in v2.4.3 i was using log level 3, but i got the crash just after updating to v2.6.2 |
|
So the way it should work is the following:
The idea is to have logs written into that stream and later on convert it to a string to access. I tried replacing this constructor with the other one and it worked. Dropped investigating yesterday in the mids of the code cleanup and figuring out the nature of a crash. |
|
Also since 2.4.x there’s been a major refactor of the classes to be compatible with the modern PDAL versions, prior to the change logs collection was done via executor. |
|
good you just solved it so. I've added a little guide in #81 that resume this issue |
|
Well it’s not solved yet, any help with debugging is really appreciated! Need still to figure out why it’s unhappy about the stringstream. |
|
never coded in c unfortunately. I'm trying to see but it's hard Idk if it can help but running it inside a docker went though this in particular |
I'm trying to publish my local natives inside a jar, is not my first time in doing that, but this is going to be difficult.
I've found some problems:
./sbtis no more a valid command. [Solution] Now mac users have to runbrew install sbtand then for examplesbt native/nativeCompilescripts/publish-local-native.shwill incur into./scripts/crosscompile-linux.sh: Permission denied[Solution]chmod +x ./scripts/crosscompile-linux.shpublish-local-native.shit goes intoI found two solutions for the
./sbt no such fileerror. The first is to take back in the repo the old sbt file from old versions. The second one is to modify thecrosscompile-linux.shin order to install sbt and executesbtinstead of./sbtBoth solution goes into the next error
The text was updated successfully, but these errors were encountered: