Sometimes when a new bucket is created, it can not be accessed, giving the error of Access Denied. Inside the AWS console, the bucket seems to be created properly, with no additional limiting access policies, so there is no obvious reason for this restriction. After a few hours, or restarting JupyterLab a few times, the bucket becomes accessible.
Important to note, this issue does not seem to occur at all when the bucket is created within the us-east-1 region.