Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce DBAction interface and refactoring of column dropping #709

Merged
merged 3 commits into from
Mar 25, 2025

Conversation

kvch
Copy link
Contributor

@kvch kvch commented Mar 4, 2025

This PR is part of a refactoring effort. My goal is to unify all database actions pgroll is executing. Example actions include dropping columns, renaming columns, creating and dropping triggers, creating constraints, etc.

We can define a list of these actions in a migration phase and run them sequentially. The following example is a Complete phase of an imaginary migration:

func Complete(ctx context.Context) error {
    actions := []DBAction{
        NewDropColumnAction(conn, tableName, columnName),
        NewRenameColumnAction(conn, tableName, newName, oldName),
        NewDropColumnAction(conn, tableName, CNeedsBackfill),
        NewDropTriggerAction(conn, triggerName),
    }

    for _, action := actions {
       if err := action.Execute(ctx); err != nil {
           return err
       }
    }
}

In this step I implemented column dropping as an action. In later PRs I would like to add creating and dropping triggers, functions, etc.

@github-actions github-actions bot temporarily deployed to Docs Preview March 4, 2025 15:24 Inactive
@kvch kvch changed the title Introduce DBAction interface and refactoring of column dropping Introduce DBAction interface and refactoring of column dropping Mar 4, 2025
@github-actions github-actions bot temporarily deployed to Docs Preview March 4, 2025 15:26 Inactive
@kvch kvch marked this pull request as draft March 4, 2025 15:32
@github-actions github-actions bot temporarily deployed to Docs Preview March 4, 2025 15:35 Inactive
@kvch kvch marked this pull request as ready for review March 4, 2025 15:38
@kvch kvch requested a review from exekias March 4, 2025 15:38
@exekias
Copy link
Member

exekias commented Mar 6, 2025

@kvch code change LGTM but I would wait on the rest of the team to see if this is the desired path! my personal view:

I kind of like that operations are self-contained for the most part, with some helper functions for complex stuff like backfills. This pattern would force us to write the statements elsewhere, hiding the operation implementation a bit. Not a big deal for me, but still...

@kvch
Copy link
Contributor Author

kvch commented Mar 7, 2025

The operations are getting less and less self-contained, because we are adding more and more complexity as go beyond the naive implementation of each migration. Also, some of the operations have overlap like create_constraint and create_table or adding a unique constraint and adding a unique index. Just to list a few examples.

Furthermore, we already have createUniqueIndexConcurrently, alterSequenceOwner, RenameColumn etc. These are already actions, and we are organically going towards this architecture. But instead of building an "accidental" architecture, I would like to preserve the clarity of pgroll and consciously prepare the codebase for the future. The operations we perform in each migration phase fits perfectly the command design pattern. This is my attempt to create reusable "commands"/"actions" without the fluff. Thus, we can focus on putting together a migration from existing building blocks, instead of concatenating the same SQL strings in multiple operations.

@exekias
Copy link
Member

exekias commented Mar 7, 2025

👍 thank you for the detailed explanation! no strong opinion on my side, to me whatever @andrew-farries and you decide works for me!

Copy link
Collaborator

@andrew-farries andrew-farries left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with the direction of this change but I think we can go further:

For at least the Start phase, and possibly also for Complete and Rollback the method could return the list of DBActions to be executed, rather than having the methods execute the actions directly.

This would allow us to more easily re-order and squash steps in multi-operation migations, for example allowing two operations to modify the same column - this is currently unsupported as each operation independently tries to duplicate the column.

WDYT @kvch ?

@andrew-farries andrew-farries requested a review from Copilot March 21, 2025 15:27

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a new DBAction interface with its dropColumnAction implementation to streamline and unify various database migration operations. The changes refactor multiple migration operations to use NewDropColumnAction for dropping columns, which improves consistency and modularity in how migration steps are executed.

  • Introduces a DBAction interface and dropColumnAction for column dropping.
  • Refactors several migration operations (add, drop, alter, and constraint operations) to use NewDropColumnAction.
  • Enhances consistency in executing SQL column drop operations across different migration files.

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.

Show a summary per file
File Description
pkg/migrations/dbactions.go New DBAction interface and dropColumnAction implementation
pkg/migrations/op_add_column.go Replaced direct ExecContext call with dropColumnAction usage
pkg/migrations/op_drop_constraint.go Refactored drop column logic to use dropColumnAction
pkg/migrations/op_create_constraint.go Updated drop column and backfill removal with dropColumnAction
pkg/migrations/op_drop_column.go Refactored drop column steps to use dropColumnAction
pkg/migrations/op_drop_multicolumn_constraint.go Updated multiple column drop and backfill removal with dropColumnAction
pkg/migrations/op_alter_column.go Replaced direct drops with dropColumnAction usage in complete and rollback steps
@kvch
Copy link
Contributor Author

kvch commented Mar 24, 2025

I like this approach. To keep the review burden and the changeset small, I would like to transform all existing database actions to the common format in different PRs. Then in a separate PR I will implement the new interface for Start, Complete and Rollback. Do that work for you?

@kvch kvch mentioned this pull request Mar 24, 2025
13 tasks
@andrew-farries
Copy link
Collaborator

That works for me 👍

@kvch kvch merged commit 33f9150 into xataio:main Mar 25, 2025
29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants