Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect intercept returned for logistic regression #1

Open
ff1201 opened this issue May 5, 2023 · 0 comments
Open

Incorrect intercept returned for logistic regression #1

ff1201 opened this issue May 5, 2023 · 0 comments
Assignees
Labels
bug Something isn't working

Comments

@ff1201
Copy link
Owner

ff1201 commented May 5, 2023

Using fit_sgs() with intercept=TRUE and type="logistic" yields incorrect estimates for the intercept, despite correct parameter estimates. Reproducible example, comparing it with the built-in glm() function:

set.seed(3)
n=200
p=10
X <- as.matrix(rnorm_multi(n=n,vars=p,mu=0,sd=1,r=0))
y <- 1/(1+exp(-(X %*%rnorm(p,mean=0,sd=sqrt(10)) + rnorm(n,mean=0,sd=4))))
y <- ifelse(y>0.5,1,0)

groups = 1:p
glm_fit <- glm(y ~ as.matrix(X),family="binomial")
sgs = fit_sgs(X=X,y=y, groups=groups, type="logistic", lambda=0, alpha=1, vFDR=0.1, gFDR=0.1,intercept=TRUE,standardise="none")
cbind(glm_fit$coefficients,sgs$beta)

expect_equivalent(coef(glm_fit)[-1],
                  as.matrix(sgs$beta)[-1],
                  tol = 1e-3
)

Compared to glmnet(), the intercept is closer but still slightly off (although the parameter estimates are again correct):

lasso <- glmnet(X, y, lambda = 0.1, standardize = FALSE,family="binomial",intercept=TRUE)
sgs_2 = fit_sgs(X=X,y=y, groups=groups, type="logistic", lambda=1, alpha=1, vFDR=0.1, gFDR=0.1,standardise="none",intercept=TRUE,w_weights = rep(0,p),v_weights = rep(0.1,p),tol=1e-5)

cbind(as.vector(sgs_2$beta),as.vector(rbind(lasso$a0, lasso$beta)))

Currently, the intercept is fitted alongside the other parameters but has no penalisation applied to it. In contrast to the Gaussian case, where the fitting process takes place without the intercept and it is calculated at the end.

Additionally, slower convergence is observed when using an intercept for logistic regression.

@ff1201 ff1201 added the bug Something isn't working label May 5, 2023
@ff1201 ff1201 self-assigned this May 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant