Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

netrc: Implement .netrc parsing #244

Open
wants to merge 20 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/Makefile.am
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ axel_SOURCES = \
src/ftp.h \
src/http.c \
src/http.h \
src/netrc.c \
src/netrc.h \
src/search.c \
src/search.h \
src/ssl.c \
Expand Down
3 changes: 3 additions & 0 deletions src/conf.h
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,8 @@
#ifndef AXEL_CONF_H
#define AXEL_CONF_H

#include "netrc.h"

typedef struct {
char default_filename[MAX_STRING];
char http_proxy[MAX_STRING];
Expand All @@ -67,6 +69,7 @@ typedef struct {
int search_threads;
int search_amount;
int search_top;
netrc_t *netrc;

unsigned io_timeout;

Expand Down
6 changes: 6 additions & 0 deletions src/conn.c
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@
/* Connection stuff */

#include "axel.h"
#include "netrc.h"

/**
* Convert an URL to a conn_t structure.
Expand Down Expand Up @@ -253,6 +254,11 @@ conn_init(conn_t *conn)
conn->ftp->local_if = conn->local_if;
conn->ftp->ftp_mode = FTP_PASSIVE;
conn->ftp->tcp.ai_family = conn->conf->ai_family;
if (conn->conf->netrc) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code here shouldn't be conditional, and it should be for all protocols.

We want to support other formats, like authinfo, in the future.

It should be into a separate function, so that refactoring doesn't touch code around it:

conf_auth_setup(conf);

Also auto-login should only happen if we didn't got the credentials from somewhere else, so a check for that is needed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, I will move this code to the conn.c:conn_set() function, as that is the one that deals with parsing URLs and extracting fields like username, password and hostname from them and copying to conn_t. But once parsing is entangled with auth code, I'd rather leave this kind of refactoring to another PR.

netrc_parse(conn->conf->netrc, conn->host, conn->user, sizeof(conn->user), conn->pass, sizeof(conn->pass));
netrc_close(conn->conf->netrc);
printf("host: %s, user: %s, pass: %s\n", conn->host, conn->user, conn->pass);
}
if (!ftp_connect(conn->ftp, conn->proto, conn->host, conn->port,
conn->user, conn->pass,
conn->conf->io_timeout)) {
Expand Down
209 changes: 209 additions & 0 deletions src/netrc.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,209 @@
/*
Axel -- A lighter download accelerator for Linux and other Unices

Copyright 2019 David da Silva Polverari

This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License
as published by the Free Software Foundation; either version 2
of the License, or (at your option) any later version.

In addition, as a special exception, the copyright holders give
permission to link the code of portions of this program with the
OpenSSL library under certain conditions as described in each
individual source file, and distribute linked combinations including
the two.

You must obey the GNU General Public License in all respects for all
of the code used other than OpenSSL. If you modify file(s) with this
exception, you may extend this exception to your version of the
file(s), but you are not obligated to do so. If you do not wish to do
so, delete this exception statement from your version. If you delete
this exception statement from all source files in the program, then
also delete it here.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*/

/* .netrc parsing implementation */

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/stat.h>
#include <sys/mman.h>
#include "axel.h"
#include "netrc.h"

typedef struct {
char *data;
size_t len;
} buffer_t;

static buffer_t tok, save_buf;
static const char *tok_delim = " \t\n";

static size_t
memspn(const char *s, const char *accept, size_t len)
{
size_t sz = 0;

while (*s && len-- && strchr(accept, *s++))
sz++;
return sz;
}

static size_t
memcspn(const char *s, const char *reject, size_t len)
{
size_t sz = 0;

while (*s && len--)
if (strchr(reject, *s))
return sz;
else
s++, sz++;
return sz;
}

static buffer_t
memtok(const char *addr, size_t len, const char *delim, buffer_t *save_ptr)
{
size_t sz;
char *p, *q;
buffer_t ret;

if (!addr) {
p = save_ptr->data;
} else {
p = (char *) addr;
save_ptr->len = len;
}
sz = memspn(p, delim, save_ptr->len);
p += sz;
save_ptr->len -= sz;
q = p;
sz = memcspn(q, delim, save_ptr->len);
q += sz;
save_ptr->data = q;
ret.len = q - p;
ret.data = p;
return ret;
}

static size_t
file_size(int fd)
{
struct stat st;
fstat(fd, &st);
return st.st_size;
}

static size_t
netrc_mmap(const char *filename, char **addr)
{
int fd;
size_t sz;
char *aux = NULL;
char *home = NULL;
char *path = NULL;
const char suffix[] = "/.netrc";
davidpolverari marked this conversation as resolved.
Show resolved Hide resolved

if (filename && *filename) {
aux = path = (char *)filename;
} else if ((path = getenv("NETRC"))) {
aux = path;
} else if ((home = getenv("HOME"))) {
size_t i = strlen(home);
if ((path = malloc(i + sizeof(suffix)))) {
memcpy(path, home, i);
memcpy(path+i, suffix, sizeof(suffix));
}
}
if (!path)
return 0;
fd = open(path, O_RDONLY,0);
if (!aux)
free(path);
if (fd == -1) {
return 0;
}
sz = file_size(fd);
*addr = mmap(NULL, sz, PROT_READ, MAP_PRIVATE | MAP_POPULATE, fd, 0);
close(fd);
return sz;
}

static void
get_creds(netrc_t *netrc, char *user, size_t user_len, char *pass, size_t pass_len)
{
while (tok.len) {
if (!strncmp("login", tok.data, tok.len)) {
tok = memtok(NULL, 0, tok_delim, &save_buf);
if (tok.len <= user_len)
strlcpy(user, tok.data, tok.len+1);
} else if (!strncmp("password", tok.data, tok.len)) {
tok = memtok(NULL, 0, tok_delim, &save_buf);
if (tok.len <= pass_len)
strlcpy(pass, tok.data, tok.len+1);
} else if (!strncmp("machine", tok.data, tok.len) || !strncmp("default", tok.data, tok.len)) {
save_buf.data -= tok.len;
save_buf.len += tok.len;
break;
}
tok = memtok(NULL, 0, tok_delim, &save_buf);
}
}

netrc_t *
netrc_init(const char *file)
{
netrc_t *netrc;

netrc = calloc(1, sizeof(netrc_t));
if (!netrc)
return NULL;
netrc->file = file;
davidpolverari marked this conversation as resolved.
Show resolved Hide resolved
return netrc;
}

int
davidpolverari marked this conversation as resolved.
Show resolved Hide resolved
netrc_parse(netrc_t *netrc, const char *host, char *user, size_t user_len, char *pass, size_t pass_len)
{
davidpolverari marked this conversation as resolved.
Show resolved Hide resolved
size_t sz;
char *s_addr = NULL;

if (!(sz = netrc_mmap(netrc->file, &s_addr)))
return 0;
tok = memtok(s_addr, sz, tok_delim, &save_buf);
while (tok.len) {
if (!strncmp("default", tok.data, tok.len)) {
get_creds(netrc, user, user_len, pass, pass_len);
break;
} else if (!strncmp("machine", tok.data, tok.len)) {
tok = memtok(NULL, 0, tok_delim, &save_buf);
if ((tok.len && !strncmp(host, tok.data, tok.len))) {
get_creds(netrc, user, user_len, pass, pass_len);
break;
}
}
tok = memtok(NULL, 0, tok_delim, &save_buf);
}
munmap(s_addr, sz);
return 1;
}

void netrc_close(netrc_t *netrc)
{
if (netrc)
free(netrc);
}
48 changes: 48 additions & 0 deletions src/netrc.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
/*
Axel -- A lighter download accelerator for Linux and other Unices

Copyright 2019 David da Silva Polverari

This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License
as published by the Free Software Foundation; either version 2
of the License, or (at your option) any later version.

In addition, as a special exception, the copyright holders give
permission to link the code of portions of this program with the
OpenSSL library under certain conditions as described in each
individual source file, and distribute linked combinations including
the two.

You must obey the GNU General Public License in all respects for all
of the code used other than OpenSSL. If you modify file(s) with this
exception, you may extend this exception to your version of the
file(s), but you are not obligated to do so. If you do not wish to do
so, delete this exception statement from your version. If you delete
this exception statement from all source files in the program, then
also delete it here.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*/

/* .netrc parsing include file */

#ifndef AXEL_NETRC_H
#define AXEL_NETRC_H

typedef struct {
const char *file;
} netrc_t;

netrc_t *netrc_init(const char *file);
int netrc_parse(netrc_t *netrc, const char *host, char *user, size_t user_len, char *pass, size_t pass_len);
void netrc_close(netrc_t *netrc);

#endif /* AXEL_NETRC_H */
14 changes: 13 additions & 1 deletion src/text.c
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ static struct option axel_options[] = {
{"max-redirect", 1, NULL, MAX_REDIR_OPT},
{"output", 1, NULL, 'o'},
{"search", 2, NULL, 'S'},
{"netrc", 2, NULL, 'R'},
{"ipv4", 0, NULL, '4'},
{"ipv6", 0, NULL, '6'},
{"no-proxy", 0, NULL, 'N'},
Expand Down Expand Up @@ -117,7 +118,7 @@ main(int argc, char *argv[])
j = -1;
while (1) {
int option = getopt_long(argc, argv,
"s:n:o:S::46NqvhVakcH:U:T:",
"s:n:o:S::R::46NqvhVakcH:U:T:",
axel_options, NULL);
if (option == -1)
break;
Expand Down Expand Up @@ -161,6 +162,15 @@ main(int argc, char *argv[])
}
}
break;
case 'R':
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's define a negative version, it's necessary for if there's a default in the axel config file. I'm not sure a short version is needed (I've been thinking about removing support for systems not supporting getopt_long anyway).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

About negative version of the option, I don't know whether it is a good idea to make .netrc parsing default, as it may break existing users' assumptions/scripts/workflows, etc. What do you think about it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't mean to make it the default, but if you have the option on the configuration file, then you need a way to disable it from the command line.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I'll add it.

{
char *netrc_filename = NULL;
if (optarg) {
netrc_filename = optarg;
}
conf->netrc = netrc_init(netrc_filename);
davidpolverari marked this conversation as resolved.
Show resolved Hide resolved
}
break;
case '6':
conf->ai_family = AF_INET6;
break;
Expand Down Expand Up @@ -659,6 +669,7 @@ print_help(void)
"-n x\tSpecify maximum number of connections\n"
"-o f\tSpecify local output file\n"
"-S[n]\tSearch for mirrors and download from n servers\n"
"-R[file]\tRetrieve credentials from $HOME/.netrc file or filename\n"
"-4\tUse the IPv4 protocol\n"
"-6\tUse the IPv6 protocol\n"
"-H x\tAdd HTTP header string\n"
Expand All @@ -682,6 +693,7 @@ print_help(void)
"--max-redirect=x\t\tSpecify maximum number of redirections\n"
"--output=f\t\t-o f\tSpecify local output file\n"
"--search[=n]\t\t-S[n]\tSearch for mirrors and download from n servers\n"
"--netrc[=file]\t\t-R[file]\tRetrieve credentials from $HOME/.netrc file or filename\n"
"--ipv4\t\t\t-4\tUse the IPv4 protocol\n"
"--ipv6\t\t\t-6\tUse the IPv6 protocol\n"
"--header=x\t\t-H x\tAdd HTTP header string\n"
Expand Down