Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gh-3246: Add an explain step and GafferPop to REST API #3248

Merged
merged 20 commits into from
Jul 17, 2024

Conversation

tb06904
Copy link
Member

@tb06904 tb06904 commented Jul 9, 2024

This integrates the GafferPop library into the spring boot API so that the /gremlin endpoint acts like a standard Gremlin server. This involves making a websocket endpoint available that can deserialise and respond to Gremlin queries.

Also add two new endpoints that will accept either a Gremlin or Cypher query and give an explanation as to what Gaffer operations are ran when the query is executed. They are at: /rest/gremlin/explain and /rest/gremlin/cypher/explain.

Slight overhaul to the user auth in the spring rest so that the HTTP headers can be correctly passed to a AbstractUserFactory for use in user creation.

Additional manual testing with gremlinpython and aws graph notebook using the v3 graphson serialiser.

Related issue

@tb06904 tb06904 requested review from cn337131 and p29876 July 9, 2024 17:18
@tb06904 tb06904 linked an issue Jul 9, 2024 that may be closed by this pull request
Copy link

codecov bot commented Jul 9, 2024

Codecov Report

Attention: Patch coverage is 83.13953% with 29 lines in your changes missing coverage. Please review.

Project coverage is 67.27%. Comparing base (717f544) to head (f390e77).

Files Patch % Lines
...q/gaffer/rest/handler/GremlinWebSocketHandler.java 81.57% 11 Missing and 3 partials ⚠️
...gchq/gaffer/rest/controller/GremlinController.java 82.97% 6 Missing and 2 partials ⚠️
...gchq/gaffer/tinkerpop/GafferPopGraphVariables.java 40.00% 2 Missing and 1 partial ⚠️
.../uk/gov/gchq/gaffer/rest/config/GremlinConfig.java 72.72% 2 Missing and 1 partial ⚠️
...affer/rest/factory/spring/AbstractUserFactory.java 75.00% 1 Missing ⚠️
Additional details and impacted files
@@              Coverage Diff              @@
##             develop    #3248      +/-   ##
=============================================
+ Coverage      67.21%   67.27%   +0.06%     
- Complexity      2570     2572       +2     
=============================================
  Files            922      926       +4     
  Lines          29647    29734      +87     
  Branches        3300     3290      -10     
=============================================
+ Hits           19928    20005      +77     
- Misses          8270     8282      +12     
+ Partials        1449     1447       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@tb06904 tb06904 requested review from l46978, rj77259 and wb36499 July 10, 2024 09:39
Copy link

@wb36499 wb36499 merged commit 33e3fa7 into develop Jul 17, 2024
28 checks passed
@wb36499 wb36499 deleted the gh-3246-add-an-explain-step-to-gafferpop branch July 17, 2024 14:17
@cn337131 cn337131 added enhancement Improvement to existing functionality/feature headliner Used to tag major items for release notes labels Aug 1, 2024
@cn337131 cn337131 added this to the v2.3.0 milestone Aug 13, 2024
@cn337131 cn337131 added tinkerpop Specific to/touches the tinkerpop module and removed enhancement Improvement to existing functionality/feature labels Aug 13, 2024
wb36499 added a commit that referenced this pull request Aug 13, 2024
* prepare for next development iteration

* prepare for next development iteration

* prepare for next development iteration

* Gh-3244 fix ci versioning (#3245)

* move to revision property

* update ci

* update ignore

* Gh-3085 Fix payloadOperation being exposed by getOperations (#3247)

* Gh-3085 Fix payloadOperation being exposed by getOperations

* headers

* add test

* Gh-3220: Standardise GafferPop Tests (#3243)

* update graph IT tests

* Rename GraphIT -> GraphTest

* create graph IT

* Create Modern Federated example graph

run graph tests on multiple graph types

* Update federated IT tests

move non-federates specific tests to graph tests

* tidying

* remove public + spotless

* sonarcloud

* sonarcloud

* Use allEdges/allEntities

* tidy

---------

Co-authored-by: wb36499 <[email protected]>

* Gh-3237 Implement DeleteElements for Map Store (#3239)

* Gh-3237 Implement DeleteElements for Map Store

* remove unused param

* spotless

* codecov and sonarcloud

* improve codecov and sonarcloud

* reduce test duplication and add to federated store

* checkstyle

* remove unused properties and improve check

* address PR comments and improve tests

* fix indent

* checkstyle

* copyright

* bump version

* address comments

* add comment

---------

Co-authored-by: cn337131 <[email protected]>
Co-authored-by: tb06904 <[email protected]>

* Gh-3238: target delete accumulo (#3249)

* Gh-3237 Implement DeleteElements for Map Store

* remove unused param

* spotless

* wip

* finish

* checkstyle

* codecov and sonarcloud

* improve codecov and sonarcloud

* reduce test duplication and add to federated store

* checkstyle

* remove unused properties and improve check

* Gh-3238 Implement DeleteElements for Accumulo

* add accumulo test to federated store

* remove commented test

* address comments

* address comments

* checkstyle

* checkstyle

* headers

* code cov

* remove public

* remove AccumuloClient

* code cov

* checkystle

* add test

---------

Co-authored-by: cn337131 <[email protected]>

* Gh-3246: Add an explain step and GafferPop to REST API (#3248)

* basic rest controller for gremlin

* basic websocket endpoint for gremlin inside the rest api

* improvements to the gremlin ws endpoint

* basic hooks for storing last chain

* Add working explain endpoint and tidy websocket

* Add open telementry reporting

* overhaul gremlin user auth and move cypher translator

* basic testing for gremlin controller

* move to utils class

* basic testing updates

* update the options parsing and testing

* test tweaks

* updates from testing

* tweaks

* spotless

* address comments

---------

Co-authored-by: wb36499 <[email protected]>

* Gh-3250: GetElementsWithinSet bug fix (#3251)

* initial testing and changes

* tidy up, remove bug test and copyright headers

* fix commented out lines

* additional federated tests

* reduce duplicate tests

* checkstyle

* add inOutType

* copyright

* fix version variable

* address comments

---------

Co-authored-by: wb36499 <[email protected]>

* Gh-3253 Fix custom types in gremlin (#3255)

* add conversions so gafferpop edges and vertexes are graphson compliant

* update testing

* update testing

* javadoc

* make sure cypher types can be used correctly

* typo

* Gh-3259: Improve release process (#3260)

* update release pipelines

* simplify

* release notes fix

* remove comment

* address comments

* Make gremlin config message clearer (#3264)

* Gh-3258: Sonarcloud Issues (#3261)

* initial improvements

* more test tidy up

* more fixes

* improve coverage

* copyright

* fix DeleteAllData test

* checkstyle

* address comments

* address comments

* checkstyle

* checkstyle

---------

Co-authored-by: wb36499 <[email protected]>

* Gh-3254: Standardise Spring REST user authentication (#3263)

* remove unnecessary controller interfaces and add http header setting

* test tweaks

* Update rest-api/spring-rest/src/main/java/uk/gov/gchq/gaffer/rest/controller/JobController.java

Co-authored-by: cn337131 <[email protected]>

---------

Co-authored-by: cn337131 <[email protected]>
Co-authored-by: wb36499 <[email protected]>

* prepare release gaffer2-2.3.0

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: tb06904 <[email protected]>
Co-authored-by: p29876 <[email protected]>
Co-authored-by: wb36499 <[email protected]>
Co-authored-by: cn337131 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
headliner Used to tag major items for release notes tinkerpop Specific to/touches the tinkerpop module
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add GafferPop and explain step to REST API
4 participants