This can be done using the Percona Toolkit's pt-online-schema-change
program: docs.percona.com/percona-toolkit/pt-online-schema-change.html
This program allows executing ALTER
for tables without locking them.
Although it is part of the Percona Toolkit, it is compatible with MySQL and AWS RDS Aurora.
The command would look like this:
pt-online-schema-change \
--alter="MODIFY COLUMN id BIGINT UNSIGNED NOT NULL AUTO_INCREMENT" \
--alter-foreign-keys-method=none \
--nocheck-unique-key-change \
D=<database>,t=<table> \
--execute