Skip to content

Commit

Permalink
Migrate config to KV data format
Browse files Browse the repository at this point in the history
- adding oauth support to MinIO browser (#8400) by @kanagaraj
- supports multi-line get/set/del for all config fields
- add support for comments, allow toggle
- add extensive validation of config before saving
- support MinIO browser to support proper claims, using STS tokens
- env support for all config parameters, legacy envs are also
  supported with all documentation now pointing to latest ENVs
- preserve accessKey/secretKey from FS mode setups
- add history support implements three APIs
  - ClearHistory
  - RestoreHistory
  - ListHistory
- add help command support for each config parameters
- all the bug fixes after migration to KV, and other bug
  fixes encountered during testing.
  • Loading branch information
harshavardhana committed Oct 23, 2019
1 parent f01d53b commit c242957
Show file tree
Hide file tree
Showing 185 changed files with 8,186 additions and 3,552 deletions.
1 change: 1 addition & 0 deletions .github/logo.svg
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion .travis.yml
Expand Up @@ -46,7 +46,7 @@ matrix:
go: 1.13.x
script:
- go build --ldflags="$(go run buildscripts/gen-ldflags.go)" -o %GOPATH%\bin\minio.exe
- CGO_ENABLED=1 go test -v --timeout 20m ./...
- go test -v --timeout 20m ./...

before_script:
# Add an IPv6 config - see the corresponding Travis issue
Expand Down
887 changes: 869 additions & 18 deletions CREDITS

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion Dockerfile
Expand Up @@ -17,7 +17,7 @@ FROM alpine:3.9
ENV MINIO_UPDATE off
ENV MINIO_ACCESS_KEY_FILE=access_key \
MINIO_SECRET_KEY_FILE=secret_key \
MINIO_SSE_MASTER_KEY_FILE=sse_master_key
MINIO_KMS_MASTER_KEY_FILE=kms_master_key

EXPOSE 9000

Expand Down
2 changes: 1 addition & 1 deletion Dockerfile.dev
Expand Up @@ -8,7 +8,7 @@ COPY minio /usr/bin/
ENV MINIO_UPDATE off
ENV MINIO_ACCESS_KEY_FILE=access_key \
MINIO_SECRET_KEY_FILE=secret_key \
MINIO_SSE_MASTER_KEY_FILE=sse_master_key
MINIO_KMS_MASTER_KEY_FILE=kms_master_key

RUN \
apk add --no-cache ca-certificates 'curl>7.61.0' 'su-exec>=0.2' && \
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile.release
Expand Up @@ -17,7 +17,7 @@ COPY dockerscripts/docker-entrypoint.sh /usr/bin/
ENV MINIO_UPDATE off
ENV MINIO_ACCESS_KEY_FILE=access_key \
MINIO_SECRET_KEY_FILE=secret_key \
MINIO_SSE_MASTER_KEY_FILE=sse_master_key
MINIO_KMS_MASTER_KEY_FILE=kms_master_key

RUN \
apk add --no-cache ca-certificates 'curl>7.61.0' 'su-exec>=0.2' && \
Expand Down
17 changes: 10 additions & 7 deletions README.md
@@ -1,14 +1,11 @@
# MinIO Quickstart Guide
[![Slack](https://slack.min.io/slack?type=svg)](https://slack.min.io) [![Go Report Card](https://goreportcard.com/badge/minio/minio)](https://goreportcard.com/report/minio/minio) [![Docker Pulls](https://img.shields.io/docker/pulls/minio/minio.svg?maxAge=604800)](https://hub.docker.com/r/minio/minio/)

MinIO is an object storage server released under Apache License v2.0. It is compatible[1] with Amazon S3 cloud storage service. It is best suited for storing unstructured data such as photos, videos, log files, backups and container / VM images. Size of an object can range from a few KBs to a maximum of 5TB.
[![MinIO](.github/logo.svg)](https://min.io)

MinIO server is light enough to be bundled with the application stack, similar to NodeJS, Redis and MySQL.
MinIO is an object storage server released under Apache License v2.0. It is compatible with Amazon S3 cloud storage service. It is best suited for storing unstructured data such as photos, videos, log files, backups and container / VM images. Size of an object can range from a few KBs to a maximum of 5TB.

[1] MinIO in its default mode is faster and does not calculate MD5Sum unless passed by client. This may lead to incompatibility with some S3 clients like s3ql that heavily depend on MD5Sum. For such applications start MinIO with `--compat` option.
```sh
minio --compat server /data
```
MinIO server is light enough to be bundled with the application stack, similar to NodeJS, Redis and MySQL.

## Docker Container
### Stable
Expand All @@ -22,7 +19,7 @@ docker run -p 9000:9000 minio/minio server /data
docker pull minio/minio:edge
docker run -p 9000:9000 minio/minio:edge server /data
```
Note: Docker will not display the autogenerated keys unless you start the container with the `-it`(interactive TTY) argument. Generally, it is not recommended to use autogenerated keys with containers. Please visit MinIO Docker quickstart guide for more information [here](https://docs.min.io/docs/minio-docker-quickstart-guide)
Note: Docker will not display the default keys unless you start the container with the `-it`(interactive TTY) argument. Generally, it is not recommended to use default keys with containers. Please visit MinIO Docker quickstart guide for more information [here](https://docs.min.io/docs/minio-docker-quickstart-guide)

## macOS
### Homebrew
Expand Down Expand Up @@ -189,5 +186,11 @@ mc admin update <minio alias, e.g., myminio>
## Contribute to MinIO Project
Please follow MinIO [Contributor's Guide](https://github.com/minio/minio/blob/master/CONTRIBUTING.md)

## Caveats
MinIO in its default mode doesn't use MD5Sum checkums of incoming streams unless requested by the client in `Content-Md5` header for validation. This may lead to incompatibility with rare S3 clients like `s3ql` which unfortunately do not set `Content-Md5` but depend on hex MD5Sum for the stream to be calculated by the server. MinIO considers this as a bug in `s3ql` and should be fixed on the client side because MD5Sum is a poor way to checksum and validate the authenticity of the objects. Although MinIO provides a workaround until client applications are fixed use `--compat` option instead to start the server.
```sh
./minio --compat server /data
```

## License
[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2Fminio%2Fminio.svg?type=large)](https://app.fossa.io/projects/git%2Bgithub.com%2Fminio%2Fminio?ref=badge_large)
2 changes: 2 additions & 0 deletions browser/app/js/App.js
Expand Up @@ -18,11 +18,13 @@ import React from "react"
import { Route, Switch, Redirect } from "react-router-dom"
import Browser from "./browser/Browser"
import Login from "./browser/Login"
import OpenIDLogin from "./browser/OpenIDLogin"
import web from "./web"

export const App = () => {
return (
<Switch>
<Route path={"/login/openid"} component={OpenIDLogin} />
<Route path={"/login"} component={Login} />
<Route path={"/:bucket?/*"} component={Browser} />
</Switch>
Expand Down
6 changes: 1 addition & 5 deletions browser/app/js/browser/AboutModal.js
Expand Up @@ -19,7 +19,7 @@ import { Modal } from "react-bootstrap"
import logo from "../../img/logo.svg"

export const AboutModal = ({ serverInfo, hideAbout }) => {
const { version, memory, platform, runtime } = serverInfo
const { version, platform, runtime } = serverInfo
return (
<Modal
className="modal-about modal-dark"
Expand All @@ -42,10 +42,6 @@ export const AboutModal = ({ serverInfo, hideAbout }) => {
<div>Version</div>
<small>{version}</small>
</li>
<li>
<div>Memory</div>
<small>{memory}</small>
</li>
<li>
<div>Platform</div>
<small>{platform}</small>
Expand Down
35 changes: 8 additions & 27 deletions browser/app/js/browser/ChangePasswordModal.js
Expand Up @@ -80,13 +80,6 @@ export class ChangePasswordModal extends React.Component {

generateAuth(e) {
const { serverInfo } = this.props
// Generate random access key only for root user
if (!serverInfo.userInfo.isIAMUser) {
this.setState({
newAccessKey: getRandomAccessKey()
})
}

this.setState({
newSecretKey: getRandomSecretKey(),
newSecretKeyVisible: true
Expand All @@ -100,10 +93,16 @@ export class ChangePasswordModal extends React.Component {
return false
}

// When credentials are set on ENV, password change not allowed for owner
if (serverInfo.info.isEnvCreds && !serverInfo.userInfo.isIAMUser) {
// Password change is not allowed for temporary users(STS)
if(serverInfo.userInfo.isTempUser) {
return false
}

// Password change is only allowed for regular users
if (!serverInfo.userInfo.isIAMUser) {
return false
}

return true
}

Expand Down Expand Up @@ -186,24 +185,6 @@ export class ChangePasswordModal extends React.Component {
</div>

<div className="has-toggle-password m-t-30">
{!serverInfo.userInfo.isIAMUser && (
<InputGroup
value={this.state.newAccessKey}
id="newAccessKey"
label={"New Access Key"}
name="newAccesskey"
type="text"
spellCheck="false"
required="required"
autoComplete="false"
align="ig-left"
onChange={e => {
this.setState({ newAccessKey: e.target.value })
}}
readonly={serverInfo.userInfo.isIAMUser}
/>
)}

<i
onClick={() => {
this.setState({
Expand Down
25 changes: 23 additions & 2 deletions browser/app/js/browser/Login.js
Expand Up @@ -22,14 +22,18 @@ import Alert from "../alert/Alert"
import * as actionsAlert from "../alert/actions"
import InputGroup from "./InputGroup"
import web from "../web"
import { Redirect } from "react-router-dom"
import { Redirect, Link } from "react-router-dom"
import qs from "query-string"
import storage from "local-storage-fallback"
import history from "../history"

export class Login extends React.Component {
constructor(props) {
super(props)
this.state = {
accessKey: "",
secretKey: ""
secretKey: "",
discoveryDoc: {}
}
}

Expand Down Expand Up @@ -83,6 +87,14 @@ export class Login extends React.Component {
document.body.classList.add("is-guest")
}

componentDidMount() {
web.GetDiscoveryDoc().then(({ DiscoveryDoc }) => {
this.setState({
discoveryDoc: DiscoveryDoc
})
})
}

componentWillUnmount() {
document.body.classList.remove("is-guest")
}
Expand Down Expand Up @@ -127,6 +139,15 @@ export class Login extends React.Component {
<i className="fas fa-sign-in-alt" />
</button>
</form>
{this.state.discoveryDoc &&
this.state.discoveryDoc.authorization_endpoint && (
<div className="openid-login">
<div className="or">or</div>
<a href={"/login/openid"} className="btn openid-btn">
Log in with OpenID
</a>
</div>
)}
</div>
<div className="l-footer">
<a className="lf-logo" href="">
Expand Down
170 changes: 170 additions & 0 deletions browser/app/js/browser/OpenIDLogin.js
@@ -0,0 +1,170 @@
/*
* MinIO Cloud Storage (C) 2019 MinIO, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

import React from "react"
import { connect } from "react-redux"
import logo from "../../img/logo.svg"
import Alert from "../alert/Alert"
import * as actionsAlert from "../alert/actions"
import InputGroup from "./InputGroup"
import web from "../web"
import { Redirect } from "react-router-dom"
import qs from "query-string"
import { getRandomString } from "../utils"
import storage from "local-storage-fallback"
import jwtDecode from "jwt-decode"

export class OpenIDLogin extends React.Component {
constructor(props) {
super(props)
this.state = {
clientID: "",
discoveryDoc: {}
}
this.clientIDChange = this.clientIDChange.bind(this)
this.handleSubmit = this.handleSubmit.bind(this)
}

clientIDChange(e) {
this.setState({
clientID: e.target.value
})
}

handleSubmit(event) {
event.preventDefault()
const { showAlert } = this.props
let message = ""
if (this.state.clientID === "") {
message = "Client ID cannot be empty"
}
if (message) {
showAlert("danger", message)
return
}

if (this.state.discoveryDoc && this.state.discoveryDoc.authorization_endpoint) {
const redirectURI = window.location.href.split("#")[0]
var params = new URLSearchParams()
params.set("response_type", "id_token")
params.set("scope", "openid")
params.set("client_id", this.state.clientID)
params.set("redirect_uri", redirectURI)

// Store nonce in localstorage to check again after the redirect
const nonce = getRandomString(16)
params.set("nonce", nonce)
storage.setItem("openIDKey", nonce)

const authURL = `${
this.state.discoveryDoc.authorization_endpoint
}?${params.toString()}`
window.location = authURL
}
}

componentWillMount() {
const { clearAlert } = this.props
// Clear out any stale message in the alert of previous page
clearAlert()
document.body.classList.add("is-guest")

web.GetDiscoveryDoc().then(({ DiscoveryDoc }) => {
this.setState({
discoveryDoc: DiscoveryDoc
})
})
}

componentDidMount() {
const values = qs.parse(this.props.location.hash)
if (values.error) {
this.props.showAlert("danger", values.error_description)
return
}

if (values.id_token) {
// Check nonce on the token to prevent replay attacks
const tokenJSON = jwtDecode(values.id_token)
if (storage.getItem("openIDKey") !== tokenJSON.nonce) {
this.props.showAlert("danger", "Invalid auth token")
return
}

web.LoginSTS({ token: values.id_token }).then(() => {
storage.removeItem("openIDKey")
this.forceUpdate()
return
})
}
}

componentWillUnmount() {
document.body.classList.remove("is-guest")
}

render() {
const { clearAlert, alert } = this.props
if (web.LoggedIn()) {
return <Redirect to={"/"} />
}
let alertBox = <Alert {...alert} onDismiss={clearAlert} />
// Make sure you don't show a fading out alert box on the initial web-page load.
if (!alert.message) alertBox = ""
return (
<div className="login">
{alertBox}
<div className="l-wrap">
<form onSubmit={this.handleSubmit}>
<InputGroup
value={this.state.clientID}
onChange={this.clientIDChange}
className="ig-dark"
label="Client ID"
id="clientID"
name="clientID"
type="text"
spellCheck="false"
required="required"
/>
<button className="lw-btn" type="submit">
<i className="fas fa-sign-in-alt" />
</button>
</form>
</div>
<div className="l-footer">
<a className="lf-logo" href="">
<img src={logo} alt="" />
</a>
<div className="lf-server">{window.location.host}</div>
</div>
</div>
)
}
}

const mapDispatchToProps = dispatch => {
return {
showAlert: (type, message) =>
dispatch(actionsAlert.set({ type: type, message: message })),
clearAlert: () => dispatch(actionsAlert.clear())
}
}

export default connect(
state => state,
mapDispatchToProps
)(OpenIDLogin)

0 comments on commit c242957

Please sign in to comment.