-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Beta #1
Conversation
This connector sets the fundations for this library. It needs access/secret keys, regions and eventually an aws.Config to be created. If no aws.Config is provided, then a default one will be used. Among the thing it does: * It understands 'complex' list of regions - with regex. Therefore 'eu-*' and 'us-*' can both be passed to have all those regions available. * It saves the accountID for later use - such as AMI or Volume snapshots. * It instanciates sessions for later use of services: ELB, EC2, etc. It should provide an abstraction easy enough to use all the services of AWS in multiple regions without having to bother with their multiple packages, connectors, etc.
This methods simply gets all the ELB (v1) from all the regions that have been enabled in the connector.
In order to use the library, only a couple parameters need to be set, in fact the library needs as little as 1 call to be ready to be used. A first one to get a connector object, and the other calls to directly gather information from their AWS account/regions.
In case invalid (empty list, or non-matching pattern) regions the code will now return an error with the proper reason.
The comments have been added in case we want to generate documentation via godoc, like this is the case on many other public projects. The const variable didn't need to be accessed by all the methods so moving it to the function using it was enough.
A new type of error has been created in order to save multiple errors while doing call to the API. This was done to both be able to save extra information (all errors, region & servicename), and to return the standard error type.
Those methods will allow to fetch: * Instances * VPCs * Volumes * Snapshots (only belonging to the accountID) * AMI (only belonging to the accountID) * Subnet * Security groups
In AWS SDK-go, the elbv2 represent the ELB of second generation, which are basically the ALB, but those live in another package than the first version.
In order to mock AWS calls, we have to use interfaces, otherwise we wouldn't be able to create our own methods to fake them. The modification has also been followed by the update of: svc.$SERVICE.ServiceName to $SERVICE.ServiceName As the interface doesn't have this variable, and this variable is in fact set as a const in the package, so there is no need to try to access the variable from each service.
This addition allows to fetch DBInstances as well as Tags associated with RDS resources.
The implicit convention in Go wants to have the exported methods/struct at the top of the file, while the non-exported ones at the bottom.
The methods added allow to fetch Clusters as well as tags from elasticache resources - same model as RDS. Tests have also been added with mocking of the AWS API calls, however the method 'New' couldn't be mocked, and it didn't make too much sense to mock it anyway, therefore this part is missing.
GoDoc relies on comments above struct and methods, so simple explanations have been added to every exported element.
An extra test has been added to elasticache, as 'multiple regions with error' was missing for the tags. The formatting of test messages has been revised in order to comply with: `$test-name [$index] - $element: received=%+v | expected=%+v` ``` -- FAIL: TestGetElasticacheTags (0.00s) elasticache_test.go:354: multiple region with error [3] - tags: received=[{ } { TagList: [{ Key: "test", Value: "2" }] }] | expected=[{ } { TagList: [{ Key: "test", Value: "3" }] }] FAIL ``` The method checkErrors has also been moved to another file, as it will be reused for other tests.
core/error.go
Outdated
|
||
// RawsErr type satisfies the standard error interface, thus allowing us to return an error when doing multiple call via | ||
// the go-SDK, even though multiple errors are met; which is why APIErrs save those more specific errors. | ||
type RawsErr struct { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will make a type of batch errors like this
type Errs []Err
func (errs Errs) Error() string {
var output [][]string
for _, e := range errs {
output = append(output, []string{e.Region(), e.Service()})
}
return fmt.Sprintf("%d error(s) occured: %s", len(errs), output)
}
Then we don't need the AppendError
we could just do
var errs Errs
errs = append(errs, NewAPIError(e, "the region", "the service"))
Having this, we can return Errs as nil without having to handle if the slice have values or not
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indeed, good idea - will apply
core/elbv2.go
Outdated
) | ||
|
||
// Returns a list of ELB (v2) - also known as ALB - based on the input from the different regions | ||
func (c *Connector) GetLoadBalancersV2(input *elbv2.DescribeLoadBalancersInput) ([]*elbv2.DescribeLoadBalancersOutput, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because of https://github.com/cycloidio/raws/pull/1/files#r122167049
I would return and Errs
type rather than an error
interface, it isn't a problem for the caller because Errs
satisfies the error
interface
This comment applies to all the functions that the package has now and they return RawsErr
type as error
interface
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But as Errs
satisfies error
, why returning it, rather than an error
type? As we discussed, my understanding was that it was more standard, and would simply make more sense. No?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ups, I didn't want to mean that.
For such reason, I was complaining about AWS SDK returns a generic error than their specific error interfaces on methods that they can only return one specific error type. If their function can return multiple interface type of errors in a function, you have to choose the narrowest interface which is satisfied by all of those, obviously the error
interface should be satisfied by all of them, so there is always that chance.
I would like to remember the right terminology but if I don't remember bad (I'm shit remember names and terminology), a function which is more extensible must be covariance in their arguments and contravariance in their return values (Covariance and contravariance (computer science)
What's standard is to return an value which satisfies the error
interface. What I wanted to mean is that returning type struct
rather than a type interface
has more limitations for the callers and from the creator point of view, could lead to export some struct fields
that they can be removed later on, but the callers are relying on them, so it's more prone to break compatibility than using interfaces.
For this reason, I suggest to return the interface.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe I took all notes into considerations!
The `RawsErr` did not make too much sense as we could simple defined a `type Errs []Err`, thus simplifying the adding of errors as well as the overall management/return. The naming of errors in methods has been reviewed to be `errs` everywhere, and those methods now check whether an error happened or not before adding it (same for resources), in order to have a data more representative of what really happens during calls.
The .idea repository is created by my IDE, it does not make sense to try tracking this file, as it is useless for the project.
The method is fetching DB Instances, so it doesn't make sense to store those as 'elb' while really they are 'instance'.
core/helpers_test.go
Outdated
) | ||
|
||
func checkErrors(t *testing.T, name string, index int, err error, expected error) { | ||
func checkErrors(t *testing.T, name string, index int, err Errs, expected Errs) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because it's used reflect.DeepEqual
the parameters wouldn't have needed to be changed and the function will be more generic for any test. Being a test we don't need to show any semantics in docs which types can provide
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I simply thought, that as we were using our own type, it would make more sense to use it there as well. But that would indeed work with both, I'll revert it.
core/error.go
Outdated
} | ||
|
||
// Error returns a string containing the region, service as well as the original API error message. | ||
func (e *callErr) Error() string { | ||
func (e callErr) Error() string { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's the reason behind this change? (With this change I mean the change in the type which is declared the methods required by the interface)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hum, it's a mistake from my side, as when using Errs
I couldn't have a pointer receiver for Error()
otherwise I would hit: Errs does not implement error (Error method has pointer receiver)
; and I wrongfully applied it to callErr
as well.
The receiver for callErr did not need to be updated. The checkError didn't need to be updated as both Err implement error, we can keep this generic type to test other things.
These methods allow to get buckets/objects and their tags in multiple regions. One improvement could be to provide a region for GetObjects otherwise the Connector, will try to look it up through the regions. That could be seen as both convenient and 'unecessary' as extra call are done, but the client should know which regions they manage afterall.
Basic tests have been added to the various components currently used in the project (ec2, elb, elbv2, s3 and rds). The mocking is fairly simple, as the goal is to ensure that our methods are actually returning the good set of data and/or errors. Tests regarding accountID, and such things haven't been added as the purpose is not to reproduce AWS behaviour - we trust them -, but simply what our methods are doing.
In order to make testing/resusability easier, the AWS component used to create a Connector are being isolated into 'configureAWS' function. If we realise that we need to store them, we could also easily fix the code.
The connector file is a bit trickier to test/mock because there are various calls to AWS which can't really be tested/mocked ('New'), but because they have been gathered in one function, the rest had be tested.
The error declares couple struct/interface/methods which needed to be tested as well, to validate the different output we would have by calling them.
The package was located in a subdir, which didn't make much sense for a library like this one. Leaving everything at the root is more go-idomatic.
In order to provide a simple example the main.go has been moved to an 'example' directoy, rather than being left in the root, and give the impression that it is a binary.
In order to have a small CI running on the library we needed to add those elements.
The default 'install' rule of travis-ci is 'true' when a Makefile is present, so in order to avoid that, it will now download the packages for the project and tests.
It is meant to start explaining what is the project, and how to use the library, as well as the current limitations or elements to keep in mind.
The README had one or two typos in the getting started, and could benefit from links for both example and license file.
Changes Unknown when pulling 0cdf330 on beta into ** on master**. |
Overall I see no errors, just some punctuations (it's a big PR hehe) For the documentation I recomend you to read this post which explains the go way.
For the code everithing is clear (I read it in an overview as its big) but I think that it's well formatted, ther are some funcions in which you could use a go syntax suggar on declare repeated types on the arguments, example you could do:
instead of
Other things are opiniated, the use of |
Thank you for the comments, I'll take a look at the doc link. Regarding the notation, for me it really depends of the situation. And on the arguments, I personally prefer to use the shortened version, only when all arguments have the same type, I find it confusing otherwise. |
I also prefer to use the long syntax to declare variables
Obviously 1 enforce to use the short syntax on such cases if you only want to declare the variable inside of the loop and conditional scope Regarding the Go syntax sugar for the types of the function arguments, I wold have prefer that it won't exist, so I prefer to use the long syntax, rather than the short, although it's something that, at the beginning I used the short syntax, later I've been doubting between both and now I prefer the long one. The case that @xlr-8 mentions could be a good case to consider to use it, however I'm still not sold on using both rather than one. As you commented this is a personal opinion, so I don't think that we should enforce one, they are minor things to remember and because Go has a short set of instructions and statements and syntax sugar tricks, every Go developer know them, so the syntax looks familiar. If you would like to enforce one, I'm not agains but only if we can detect them through a linter. |
No description provided.