Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix extractDuplicates & added resource transformers from maven-shade-plu... #614

Merged
merged 1 commit into from
May 15, 2015

Conversation

kedzie
Copy link
Contributor

@kedzie kedzie commented Mar 23, 2015

...gin

-extractDuplicates had cases where it removed all copies of duplicates. added test to verify behavior is fixed
-support for using resource transformers from maven-shade-plugin to merge duplicate resources
-added integration test for said behavior

@kedzie
Copy link
Contributor Author

kedzie commented Mar 24, 2015

The case that the current extractDuplicates functionality fails is the same as in my integration test. There are 3 libraries:

libA -> resourceA,
libB ->resourceA, resourceB
libC -> resouceB

The current functionality builds a map from filenames to the jars they are found in, so for this example that would be:

resourceA --> libA, libB
resourceB --> libB, libC

Then all duplicates resources are removed from each list of jars, except for the first jar (so one copy of the resource remains) The pro lem here is that libB appears both as the first element, for resourceB, and as the second element, for resourceA. This means it is in the list of jars from which duplicates are removed. The result is that resourceB is not found in the output apk.

My extract duplicates method resolves this issue by removed duplicates from ALL the jars in the list, then adding them back into an additional duplicate-resources.jar that is created by the plugin. This is also where the resource transformers put their output. The additional jar is added to the list of jars from which resources are added to the apk. Hope this makes it clear.

@@ -265,6 +265,12 @@
<version>2.2</version>
</dependency>

<dependency>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a whole other plugin as a dependency seems like a bad idea. Why is this needed? Just for the ResourceTransformer?

@kedzie
Copy link
Contributor Author

kedzie commented Mar 24, 2015

Yes just for the ResourceTransformer interface and implementations. My logic was to reuse the existing transformers instead of recreating them. So that any other 3rd party transformers could be reused. What do you suggest instead of adding the dependency? Copying the source files? I'm open to whatever you think is best.

@kedzie kedzie force-pushed the master branch 2 times, most recently from 754612f to f1962bb Compare March 24, 2015 22:28
@mosabua
Copy link
Member

mosabua commented Apr 14, 2015

This fails on the IT tests for me... can you try a full build and see if you can fix this?

@mosabua
Copy link
Member

mosabua commented Apr 14, 2015

Running com.jayway.maven.plugins.android.sample.TicTacToeSampleIT
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 7.289 sec <<< FAILURE! - in com.jayway.maven.plugins.android.sample.TicTacToeSampleIT
buildDeployAndRun3.2.3 Time elapsed: 7.288 sec <<< FAILURE!
java.lang.AssertionError: [[ERROR] Failed to execute goal com.simpligility.maven.plugins:android-maven-plugin:4.1.2-SNAPSHOT:manifest-merger (manifestMerger) on project tictactoe-app: Execution manifestMerger of goal com.simpligility.maven.plugins:android-maven-plugin:4.1.2-SNAPSHOT:manifest-merger failed: com.android.manifmerger.ManifestMerger2$MergeFailureException: org.xml.sax.SAXNotRecognizedException: Feature 'http://xml.org/sax/features/xmlns-uris' is not recognized. -> [Help 1], [ERROR] , [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch., [ERROR] Re-run Maven using the -X switch to enable full debug logging., [ERROR] , [ERROR] For more information about the errors and possible solutions, please read the following articles:, [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException, [ERROR] , [ERROR] After correcting the problems, you can resume the build with the command, [ERROR] mvn -rf :tictactoe-app]
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at io.takari.maven.testing.executor.MavenExecutionResult.assertErrorFreeLog(MavenExecutionResult.java:44)
at com.jayway.maven.plugins.android.sample.TicTacToeSampleIT.buildDeployAndRun(TicTacToeSampleIT.java:58)

Results :

Failed tests:
TicTacToeSampleIT.buildDeployAndRun:58 [[ERROR] Failed to execute goal com.simpligility.maven.plugins:android-maven-plugin:4.1.2-SNAPSHOT:manifest-merger (manifestMerger) on project tictactoe-app: Execution manifestMerger of goal com.simpligility.maven.plugins:android-maven-plugin:4.1.2-SNAPSHOT:manifest-merger failed: com.android.manifmerger.ManifestMerger2$MergeFailureException: org.xml.sax.SAXNotRecognizedException: Feature 'http://xml.org/sax/features/xmlns-uris' is not recognized. -> [Help 1], [ERROR] , [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch., [ERROR] Re-run Maven using the -X switch to enable full debug logging., [ERROR] , [ERROR] For more information about the errors and possible solutions, please read the following articles:, [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException, [ERROR] , [ERROR] After correcting the problems, you can resume the build with the command, [ERROR] mvn -rf :tictactoe-app]

Tests run: 12, Failures: 1, Errors: 0, Skipped: 1

[WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent! The file encoding for reports output files should be provided by the POM property ${project.reporting.outputEncoding}.
[INFO]
[INFO] --- maven-failsafe-plugin:2.18.1:verify (default) @ android-maven-plugin ---
[INFO] Failsafe report directory: /Volumes/mac-data/dev/github/simpligility/android-maven-plugin/target/failsafe-reports
[WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent! The file encoding for reports output files should be provided by the POM property ${project.reporting.outputEncoding}.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 06:54 min
[INFO] Finished at: 2015-04-13T23:15:13-07:00
[INFO] Final Memory: 63M/428M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-failsafe-plugin:2.18.1:verify (default) on project android-maven-plugin: There are test failures.
[ERROR]
[ERROR] Please refer to /Volumes/mac-data/dev/github/simpligility/android-maven-plugin/target/failsafe-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

@kedzie
Copy link
Contributor Author

kedzie commented May 3, 2015

@mosabua Fixed the tests. Seems like some transitive dependencies from maven-shade-plugin were interfering. I excluded them. There was another failure in the aar-child test project, which I fixed by changing the junit version.

Finally, what is final word on the maven-shade-plugin dependency? Do you think it is better to include the ResourceTransformer source code?

@kedzie
Copy link
Contributor Author

kedzie commented May 3, 2015

This change also fixes a bug in extractDuplicates. The jar files which are created during the duplicate extraction are created in the project's outputDirectory (i.e. /target/classes); which means they end up getting packaged in the APK artifact, which makes no sense.

@mosabua
Copy link
Member

mosabua commented May 7, 2015

I am fine with pulling this in, but I have to have a closer look first.

…plugin

-extractDuplicates had cases where it removed all copies of duplicates. added test to verify behavior is fixed
-support for using resource transformers from maven-shade-plugin to merge duplicate resources
-added integration test for said behavior
@kedzie
Copy link
Contributor Author

kedzie commented May 13, 2015

@mosabua Rebased.

@mosabua mosabua merged commit 6a1a483 into simpligility:master May 15, 2015
@mosabua
Copy link
Member

mosabua commented May 15, 2015

its in ;-)

@athkalia
Copy link
Contributor

Right after upgrading from 4.2.1 to 4.3.0 I get build failures when building my multi-module project. Getting the erros at the espresso instrumentation test module:

Here's part of the build

[INFO] --- android-maven-plugin:4.3.0:apk (default-apk) @ espresso ---
[INFO] Generating debug apk.
[INFO] Adding native libraries : []
[WARNING] Duplicate file LICENSE.txt : [C:\Users\sakis\.m2\repository\org\hamcrest\hamcrest-integration\1.1\hamcrest-integration-1.1.jar, C:\Users\sakis\.m2\repository\org\hamcrest\hamcrest-library\1.1\hamcrest-library-1.1.jar, C:\Users\sakis\.m2\repository\jun
it\junit-dep\4.10\junit-dep-4.10.jar, C:\Users\sakis\.m2\repository\org\hamcrest\hamcrest-core\1.1\hamcrest-core-1.1.jar]
[INFO] hamcrest-integration-1.1.jar rewritten without duplicates : C:\Users\sakis\Desktop\intelliJ projects\follower\espresso\target\unpacked-embedded-jars\hamcrest-integration-1.1.jar
[INFO] hamcrest-library-1.1.jar rewritten without duplicates : C:\Users\sakis\Desktop\intelliJ projects\follower\espresso\target\unpacked-embedded-jars\hamcrest-library-1.1.jar
[ERROR] Cannot removing duplicates : duplicate entry: LICENSE.txt
[ERROR] Cannot removing duplicates : duplicate entry: LICENSE.txt
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Follower ........................................... SUCCESS [  0.143 s]
[INFO] Follower - Common .................................. SUCCESS [  3.880 s]
[INFO] Follower - Application ............................. SUCCESS [05:24 min]
[INFO] Follower - Application Espresso Tests .............. FAILURE [ 18.554 s]
[INFO] Follower - Server .................................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 05:48 min
[INFO] Finished at: 2015-06-16T19:20:58+03:00
[INFO] Final Memory: 57M/971M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.simpligility.maven.plugins:android-maven-plugin:4.3.0:apk (default-apk) on project espresso: Duplicated file: LICENSE.txt, found in archive C:\Users\sakis\.m2\repository\junit\junit-dep\4.10\junit-dep-4.10.jar and C:\Users\sak
is\.m2\repository\org\hamcrest\hamcrest-core\1.1\hamcrest-core-1.1.jar: Duplicate files at the same path inside the APK -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :espresso

@mosabua
Copy link
Member

mosabua commented Jun 16, 2015

Can you file a separate new issue @athkalia that includes a test project ideally so that @kedzie can look into it?

@athkalia
Copy link
Contributor

I'll give that a go when I get some time. For now I moved back to the previous 4.2.1 version and everything works just fine.

@kedzie
Copy link
Contributor Author

kedzie commented Jun 16, 2015

Yes if u can help me recreate it I will fix it.
On Jun 16, 2015 2:37 PM, "athkalia" [email protected] wrote:

I'll give that a go when I get some time. For now I moved back to the
previous 4.2.1 version and everything works just fine.


Reply to this email directly or view it on GitHub
#614 (comment)
.

@athkalia
Copy link
Contributor

@kedzie this is a sample project that has the issue described.
https://www.dropbox.com/sh/enc8mkmakrh3a6q/AACyEEH3-0VoNMM4p_pV1uuIa?dl=0

Just "mvn clean install -DskipTests" on the parent pom.xml. It's pretty much empty, just dependencies in there.

@kedzie
Copy link
Contributor Author

kedzie commented Jul 25, 2015

I'm taking a look at it if it is still an issue. sorry for the delay. @athkalia

@kedzie
Copy link
Contributor Author

kedzie commented Jul 25, 2015

@athkalia I pulled this down and there are a bunch of missing dependencies so it does not build. I also noticed you didn't set extractDependencies in the application apk. Anyway I cannot run the build and therefore cannot recreate the issue.

[INFO] Scanning for projects...
[WARNING] The POM for com.android.support:appcompat-v7:jar:19.+ is missing, no dependency information available
[WARNING] The POM for in.srain.cube:ultra-ptr:aar:1.0.7.flw is missing, no dependency information available
[WARNING] The POM for com.github.lzyzsd:circleprogress:aar:1.1.0.flw is missing, no dependency information available
[WARNING] The POM for cn.pedant.sweetalert:library:aar:1.3.1 is missing, no dependency information available
[WARNING] The POM for com.commonsware.cwac:wakeful:jar:1.0.4 is missing, no dependency information available
Downloading: https://repository-cnh-eagledisplay.forge.cloudbees.com/snapshot/com/follower/common/0.5-SNAPSHOT/maven-metadata.xml
Downloading: https://repository-cnh-eagledisplay.forge.cloudbees.com/snapshot/com/follower/common/0.5-SNAPSHOT/common-0.5-SNAPSHOT.pom
[WARNING] The POM for com.follower:common:jar:0.5-SNAPSHOT is missing, no dependency information available
[WARNING] The POM for com.follower:common:jar:tests:0.5-SNAPSHOT is missing, no dependency information available
Downloading: https://repository-cnh-eagledisplay.forge.cloudbees.com/snapshot/com/follower/common/0.5-SNAPSHOT/common-0.5-SNAPSHOT.jar
Downloading: https://repository-cnh-eagledisplay.forge.cloudbees.com/snapshot/com/follower/common/0.5-SNAPSHOT/common-0.5-SNAPSHOT-tests.jar
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Follower - Application 0.5-SNAPSHOT
[INFO] ------------------------------------------------------------------------
Downloading: https://repository-cnh-eagledisplay.forge.cloudbees.com/snapshot/org/apache/maven/plugins/maven-install-plugin/maven-metadata.xml
Downloading: https://repository-cnh-eagledisplay.forge.cloudbees.com/snapshot/org/apache/maven/plugins/maven-deploy-plugin/maven-metadata.xml
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 7.038 s
[INFO] Finished at: 2015-07-25T15:43:52-05:00
[INFO] Final Memory: 19M/207M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project application: Could not resolve dependencies for project com.follower:application:apk:0.5-SNAPSHOT: The following artifacts could not be resolved: com.android.support:appcompat-v7:jar:19.+, in.srain.cube:ultra-ptr:aar:1.0.7.flw, com.github.lzyzsd:circleprogress:aar:1.1.0.flw, cn.pedant.sweetalert:library:aar:1.3.1, com.commonsware.cwac:wakeful:jar:1.0.4, com.follower:common:jar:0.5-SNAPSHOT, com.follower:common:jar:tests:0.5-SNAPSHOT: Failure to find com.android.support:appcompat-v7:jar:19.+ in https://repository-cnh-eagledisplay.forge.cloudbees.com/release was cached in the local repository, resolution will not be reattempted until the update interval of cloudbees-private-release-repository has elapsed or updates are forced -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException

@athkalia
Copy link
Contributor

@kedzie Sorry for that, I didnd't notice. I tried fixing the dependency problems while still re-creating the issue described above but it seems like I am stuck half way. If you try building it again there are some espresso dependencies missing. Those are

    <dependency>
            <groupId>com.android.support.test.espresso</groupId>
            <artifactId>espresso-core</artifactId>
            <type>aar</type>
        </dependency>
        <dependency>
            <groupId>com.android.support.test.espresso</groupId>
            <artifactId>espresso-contrib</artifactId>
            <type>aar</type>
        </dependency>
        <dependency>
            <groupId>com.android.support.test.espresso</groupId>
            <artifactId>espresso-intents</artifactId>
            <type>aar</type>
        </dependency>

Those are needed to recreate the issue. I've added them separately in the folder, you just need to manually add them to your .m2 directory under .m2\repository\com\android\support\test\espresso
Problem is that even when u do add them, you end up getting a weird error about the R file not existing or something.(that goes away if you connect to my nexus repo, but you don't have access to that :D ) Sorry but I can't spend any more time on this. If you feel like it, do have a look.

Also, what's that "extractDependencies " that you mentioned?

Cheers,
Sakis

@kedzie
Copy link
Contributor Author

kedzie commented Jul 27, 2015

extractDependencies is a plugin configuration parameter. If it isn't set
to true the plugin won't attempt to eliminate any duplicates. It needs to
be set in the pom android plugin configuration. You did set it in the test
project, but not in the app project.

Anyway did u think about adding a local maven repo to this example? Just
create a folder called maven_repo and add a repository element to the pom
(using file://${project.basedir}/maven_repo URI instead of http://). And
put any missing dependencies in there (with the normal repo folder
structure). So i won't need access to your nexus repo.
On Jul 26, 2015 6:55 PM, "athkalia" [email protected] wrote:

@kedzie https://github.com/kedzie Sorry for that, I didnd't notice. I
tried fixing the dependency problems while still re-creating the issue
described above but it seems like I am stuck half way. If you try building
it again there are some espresso dependencies missing. Those are

<dependency>
        <groupId>com.android.support.test.espresso</groupId>
        <artifactId>espresso-core</artifactId>
        <type>aar</type>
    </dependency>
    <dependency>
        <groupId>com.android.support.test.espresso</groupId>
        <artifactId>espresso-contrib</artifactId>
        <type>aar</type>
    </dependency>
    <dependency>
        <groupId>com.android.support.test.espresso</groupId>
        <artifactId>espresso-intents</artifactId>
        <type>aar</type>
    </dependency>

Those are needed to recreate the issue. I've added them separately in the
folder, you just need to manually add them to your .m2 directory under
.m2\repository\com\android\support\test\espresso
Problem is that even when u do add them, you end up getting a weird error
about the R file not existing or something.(that goes away if you connect
to my nexus repo, but you don't have access to that :D ) Sorry but I can't
spend any more time on this. If you feel like it, do have a look.

Also, what's that "extractDependencies " that you mentioned?

Cheers,
Sakis


Reply to this email directly or view it on GitHub
#614 (comment)
.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants