Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🎅Article Hit Backend #75

Merged
merged 64 commits into from
Feb 10, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
0bf1ba7
🐽 img zoom
iofu728 Jan 17, 2019
576b6f0
🐽 img zoom
iofu728 Jan 17, 2019
0f68723
:label: backup 2019/01/18
iofu728 Jan 17, 2019
30f812d
🙃 fix day.sh
iofu728 Jan 17, 2019
95a787b
:label: backup 2019/01/19
iofu728 Jan 18, 2019
141f10b
🤘 fix terminal
iofu728 Jan 19, 2019
50e896d
:label: backup 2019/01/20
iofu728 Jan 19, 2019
9b3a6b2
:label: backup 2019/01/21
iofu728 Jan 20, 2019
2a1c7df
:label: backup 2019/01/22
iofu728 Jan 21, 2019
d8ef390
🤘 fix terminal
iofu728 Jan 22, 2019
99a749a
🤘 fix terminal
iofu728 Jan 22, 2019
822e92c
:label: backup 2019/01/23
iofu728 Jan 22, 2019
e4d3a0c
🤘 fix terminal
iofu728 Jan 23, 2019
692f174
:label: backup 2019/01/24
iofu728 Jan 23, 2019
1bd5dee
:label: backup 2019/01/25
iofu728 Jan 24, 2019
93a517e
:label: backup 2019/01/26
iofu728 Jan 25, 2019
e9affb2
🙊 fix word
iofu728 Jan 26, 2019
c955c99
:label: backup 2019/01/27
iofu728 Jan 26, 2019
7713f0a
Merge branch 'develop' of github.com:iofu728/blog into develop
iofu728 Jan 27, 2019
636d533
:label: backup 2019/01/28
iofu728 Jan 27, 2019
cfee4f0
:label: backup 2019/01/29
iofu728 Jan 28, 2019
215f116
:label: backup 2019/01/30
iofu728 Jan 29, 2019
c4f1197
:label: backup 2019/01/31
iofu728 Jan 30, 2019
ef7b351
:label: backup 2019/02/01
iofu728 Jan 31, 2019
653d0a6
:label: backup 2019/02/02
iofu728 Feb 1, 2019
def3c4e
🐷 json-ld
iofu728 Feb 2, 2019
2594fb6
💂 init backend
iofu728 Feb 2, 2019
bc7ad86
:label: backup 2019/02/03
iofu728 Feb 2, 2019
d1f6fee
:label: backup 2019/02/04
iofu728 Feb 3, 2019
b40a847
🏮 db connect
iofu728 Feb 3, 2019
fdf3549
Merge branch 'develop' of github.com:iofu728/blog into develop
iofu728 Feb 3, 2019
52ae932
:label: backup 2019/02/05
iofu728 Feb 4, 2019
d0b4d27
🐽 mrfn,md
iofu728 Feb 5, 2019
df64460
🐽 fix mrfn
iofu728 Feb 5, 2019
c31767e
🐽 fix mrfn
iofu728 Feb 5, 2019
a6242bf
🐽 fix mrfn
iofu728 Feb 5, 2019
e26d83f
:label: backup 2019/02/06
iofu728 Feb 5, 2019
7f10d00
:label: backup 2019/02/07
iofu728 Feb 6, 2019
c4ca6e7
:label: backup 2019/02/08
iofu728 Feb 7, 2019
424a3b9
:label: backup 2019/02/09
iofu728 Feb 8, 2019
cc1e57a
:label: backup 2019/02/10
iofu728 Feb 9, 2019
9007387
🤖 title views
iofu728 Feb 9, 2019
c52d1dc
🤖 title views
iofu728 Feb 9, 2019
5bd36cd
🤖 title views
iofu728 Feb 9, 2019
abf48fd
🤖 title views
iofu728 Feb 9, 2019
8b0d4a3
🤖 title views
iofu728 Feb 9, 2019
cdb2595
🤖 title views
iofu728 Feb 9, 2019
d43d37f
🤖 title views
iofu728 Feb 9, 2019
59e22d4
🤖 title views
iofu728 Feb 10, 2019
389f401
🤖 title views
iofu728 Feb 10, 2019
3995c1a
🤖 title views
iofu728 Feb 10, 2019
5653890
🤖 title views
iofu728 Feb 10, 2019
4b8aa70
🤖 title views
iofu728 Feb 10, 2019
54573ae
🤢 fix script import two time
iofu728 Feb 10, 2019
381617d
🤢 fix script import two time
iofu728 Feb 10, 2019
209810b
🤢 fix stuck build
iofu728 Feb 10, 2019
6ebfdb6
🤢 fix script import two time
iofu728 Feb 10, 2019
c0597bc
🤢 fix stuck build
iofu728 Feb 10, 2019
117ecee
🤢 fix stuck build
iofu728 Feb 10, 2019
6a452e8
👹 migration machine
iofu728 Feb 10, 2019
7d700dc
👹 migration machine
iofu728 Feb 10, 2019
306d98d
👹 migration machine
iofu728 Feb 10, 2019
770aef2
🕴 fix title views bug
iofu728 Feb 10, 2019
bea15e9
🎅 version_4.3.0
iofu728 Feb 10, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,47 @@ crontab.log

# access.log
access.log

# gradle
.gradle

*/build/
*/code/
*/out/
*/*/out/
*/logs/
*/classes/
*/*/classes/

# some . file
.DS_Store
.factorypath
.classpath
.project
.settings
.metadata
.factorypath
.recommenders
.factorypath
.springBeans
*.class
*.iml
*.ipr
*.iws
*.jar
test.html
*/test.txt

manifest.yml
MANIFEST.MF
settings.xml
application.yml
activemq-data
overridedb.*
interpolated*.xml
dependency-reduced-pom.xml
build.log
_site/

lib/
target
15 changes: 6 additions & 9 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,16 +1,13 @@
FROM node:10

WORKDIR /usr/local/www/
FROM node:11

COPY . .

#RUN npm install yarn@latest

# yarn 1.9.4 have bug for yarn install
RUN yarn

EXPOSE 8848
RUN yarn docs:build

FROM nginx

# RUN yarn doc:dev
EXPOSE 8080

CMD [ "yarn", "doc:dev" ]
COPY docs/.vuepress/dist /usr/share/nginx/html
52 changes: 48 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
* 4.0版本基于[vuepress-theme-indigo](https://github.com/yscoder/vuepress-theme-indigo)
* img zoom(v4.2)
* 资瓷Tags(4.0 开发中)
* 实时显示访问量
* 实时显示访问量 + 文章阅读量(配合backend, v4.3)
* 自定义主题 `Gothic` Style + Card Style
* 天粒度增量Load数据 数据处理由全量改为增量 减小内存CPU占用率
* 支持Gitalk评论
Expand All @@ -36,18 +36,63 @@
[Latest release 👉 ](https://github.com/iofu728/blog/releases)

```bash
# Dev
$ git clone https://github.com/iofu728/blog.git
$ cd blog
$ yarn
$ yarn doc:dev

# Before Deploy
# Deploy
$ vim script/constant.sh #Change Service Path
$ bash script/
```

### Backend
```bash
# mysql config
$ cp blog-backend/blog-repository/src/main/resources/application.yml.temple blog-backend/blog-repository/src/main/resources/application.yml

# gradle build
$ cd blog-backend
$ ./gradlew clean build -x test
$ nohup java -jar blog-collector/build/libs/blog-collector-4.3.0-SNAPSHOT.jar >> test.txt 2>&1 &
```

## 部署指南
```bash
.
├── README.md
├── blog-backend
│   ├── blog-collector
│   │   └── src
│   │   ├── main
│   │   │   ├── java
│   │   │   │   └── com.github.iofu728.blog.collector
│   │   │   │   ├── BlogCollectorApplicationContext.java
│   │   │   │   ├── bo
│   │   │   │   ├── collector
│   │   │   │   ├── consts
│   │   │   │   ├── filter
│   │   │   │   └── service
│   │   │   └── resources
│   │   └── test
│   ├── blog-repository
│   │   └── src
│   │   ├── main
│   │   │   ├── java
│   │   │   │   └── com.github.iofu728.blog.repository
│   │   │   │   ├── BlogRepositoryApplicationContext.java
│   │   │   │   ├── dataSource
│   │   │   │   ├── entity
│   │   │   │   ├── enums
│   │   │   │   ├── mapper
│   │   │   │   └── repository
│   │   │   └── resources
│   ├── build.gradle
│   ├── gradle.properties
│   ├── gradlew
│   ├── gradlew.bat
│   └── settings.gradle
├── script
│   ├── KPI.java // MapReduce prepare.java
│   ├── PersonVersion.java // MapReduce Map&Reduce.java
Expand All @@ -59,5 +104,4 @@ $ vim script/constant.sh #Change Service Path
└── └── pv.sh // pv计算及更新脚本 5s粒度
```



PS: 有一部分更新脚本放在[iofu728/spider-press](https://github.com/iofu728/spider-press)
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
app.name=blog-collector
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
spring.http.multipart.max-file-size=500MB
spring.http.multipart.max-request-size=500MB
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
server:
port: 8848
22 changes: 22 additions & 0 deletions blog-backend/blog-collector/build/resources/main/banner.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
////////////////////////////////////////////////////////////////////
// _ooOoo_ //
// o8888888o //
// 88" . "88 //
// (| ^_^ |) //
// O\ = /O //
// ____/`---'\____ //
// .' \\| |// `. //
// / \\||| : |||// \ //
// / _||||| -:- |||||- \ //
// | | \\\ - /// | | //
// | \_| ''\---/'' | | //
// \ .-\__ `-` ___/-. / //
// ___`. .' /--.--\ `. . ___ //
// ."" '< `.___\_<|>_/___.' >'"". //
// | | : `- \`.;`\ _ /`;.`/ - ` : | | //
// \ \ `-. \_ __\ /__ _/ .-` / / //
// ========`-.____`-.___\_____/___.-`____.-'======== //
// `=---=' //
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ //
// 佛祖保佑 永不宕机 永无BUG //
////////////////////////////////////////////////////////////////////
6 changes: 6 additions & 0 deletions blog-backend/blog-collector/build/resources/main/ehcache.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>

<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">


</ehcache>
39 changes: 39 additions & 0 deletions blog-backend/blog-collector/build/resources/main/log4j2.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN" shutdownHook="disable">
<Properties>
<Property name="PID">????</Property>
<Property name="APP_NAME">taishan-api</Property>
<Property name="LOG_EXCEPTION_CONVERSION_WORD">%xwEx{10}</Property>
<Property name="LOG_LEVEL_PATTERN">%5p</Property>
<Property name="LOG_PATTERN">${hostName} ${APP_NAME} %d{yyyy-MM-dd HH:mm:ss.SSS} ${LOG_LEVEL_PATTERN} %-40.40c{1.} %m%n${LOG_EXCEPTION_CONVERSION_WORD}</Property>
<Property name="LOG_PATH">./logs</Property>
</Properties>
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="${LOG_PATTERN}" charset="utf-8"/>
</Console>
<CatAppender name="CatAppender"/>
<Async name="AsyncFile" bufferSize="81920" blocking="false">
<AppenderRef ref="Console"/>
</Async>
</Appenders>
<Loggers>
<Logger name="org.apache.catalina.startup.DigesterFactory" level="error"/>
<Logger name="org.apache.catalina.util.LifecycleBase" level="error"/>
<Logger name="org.apache.coyote.http11.Http11NioProtocol" level="warn"/>
<logger name="org.apache.sshd.common.util.SecurityUtils" level="warn"/>
<Logger name="org.apache.tomcat.util.net.NioSelectorPool" level="warn"/>
<Logger name="org.crsh.plugin" level="warn"/>
<logger name="org.crsh.ssh" level="warn"/>
<Logger name="org.eclipse.jetty.util.component.AbstractLifeCycle" level="error"/>
<Logger name="org.hibernate.validator.internal.util.Version" level="warn"/>
<logger name="org.springframework.boot.actuate.autoconfigure.CrshAutoConfiguration" level="warn"/>
<logger name="org.springframework.boot.actuate.endpoint.jmx" level="warn"/>
<logger name="metrics_influxdb.measurements" level="fatal"/>
<logger name="org.thymeleaf" level="warn"/>
<Root level="info">
<AppenderRef ref="AsyncFile"/>
<AppenderRef ref="CatAppender"/>
</Root>
</Loggers>
</Configuration>
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
server:
port: 8848
39 changes: 39 additions & 0 deletions blog-backend/blog-collector/out/production/resources/log4j2.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN" shutdownHook="disable">
<Properties>
<Property name="PID">????</Property>
<Property name="APP_NAME">taishan-api</Property>
<Property name="LOG_EXCEPTION_CONVERSION_WORD">%xwEx{10}</Property>
<Property name="LOG_LEVEL_PATTERN">%5p</Property>
<Property name="LOG_PATTERN">${hostName} ${APP_NAME} %d{yyyy-MM-dd HH:mm:ss.SSS} ${LOG_LEVEL_PATTERN} %-40.40c{1.} %m%n${LOG_EXCEPTION_CONVERSION_WORD}</Property>
<Property name="LOG_PATH">./logs</Property>
</Properties>
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="${LOG_PATTERN}" charset="utf-8"/>
</Console>
<CatAppender name="CatAppender"/>
<Async name="AsyncFile" bufferSize="81920" blocking="false">
<AppenderRef ref="Console"/>
</Async>
</Appenders>
<Loggers>
<Logger name="org.apache.catalina.startup.DigesterFactory" level="error"/>
<Logger name="org.apache.catalina.util.LifecycleBase" level="error"/>
<Logger name="org.apache.coyote.http11.Http11NioProtocol" level="warn"/>
<logger name="org.apache.sshd.common.util.SecurityUtils" level="warn"/>
<Logger name="org.apache.tomcat.util.net.NioSelectorPool" level="warn"/>
<Logger name="org.crsh.plugin" level="warn"/>
<logger name="org.crsh.ssh" level="warn"/>
<Logger name="org.eclipse.jetty.util.component.AbstractLifeCycle" level="error"/>
<Logger name="org.hibernate.validator.internal.util.Version" level="warn"/>
<logger name="org.springframework.boot.actuate.autoconfigure.CrshAutoConfiguration" level="warn"/>
<logger name="org.springframework.boot.actuate.endpoint.jmx" level="warn"/>
<logger name="metrics_influxdb.measurements" level="fatal"/>
<logger name="org.thymeleaf" level="warn"/>
<Root level="info">
<AppenderRef ref="AsyncFile"/>
<AppenderRef ref="CatAppender"/>
</Root>
</Loggers>
</Configuration>
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
package com.github.iofu728.blog.collector;

import com.github.iofu728.blog.repository.BlogRepositoryApplicationContext;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cache.annotation.EnableCaching;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Import;
import org.springframework.scheduling.annotation.EnableScheduling;

/**
* Main class
*
* @author gunjianpan
* @date 19-02-01
*/
@SpringBootApplication
@Import(value = {
BlogRepositoryApplicationContext.class
})
@EnableCaching
@EnableScheduling
@Configuration
@ComponentScan(value = "com.github.iofu728.blog")
public class BlogCollectorApplicationContext {

public static void main(String[] args) {
SpringApplication.run(BlogCollectorApplicationContext.class, args);
}
}
Loading