Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validating Upload File for Immediate Publish #11

Merged
merged 30 commits into from
Apr 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
b15c9c0
video upload fix
sagarkothari88 Feb 27, 2024
e4648f6
Merge branch 'main' of https://github.com/spknetwork/acela-core into …
sagarkothari88 Feb 28, 2024
e5da807
video upload bug fixes
sagarkothari88 Feb 28, 2024
67aed73
validating file type after upload
sagarkothari88 Feb 29, 2024
e39d011
reverting tsd upload-dir
sagarkothari88 Feb 29, 2024
32b00d7
reverting unwanted changes
sagarkothari88 Feb 29, 2024
2364053
reverting unwanted changes
sagarkothari88 Feb 29, 2024
db59c3e
reverting unwanted changes
sagarkothari88 Feb 29, 2024
f1d6d06
reverting unwanted changes
sagarkothari88 Feb 29, 2024
61bd99f
reverting unwanted changes
sagarkothari88 Feb 29, 2024
00d5b2e
reverting unwanted changes
sagarkothari88 Feb 29, 2024
c6085d1
work in progress
sagarkothari88 Feb 29, 2024
042ef73
thumbnail upload working
sagarkothari88 Mar 2, 2024
e7c62d7
possible implementation of immediate publish
sagarkothari88 Mar 3, 2024
c91982d
upload not working
sagarkothari88 Mar 3, 2024
c856245
mock publish working
sagarkothari88 Mar 4, 2024
6d21257
Merge branch 'main' into feature/validate-file-type
Mar 5, 2024
a1d11f3
added checks
sagarkothari88 Mar 8, 2024
a88248d
using env variable instead of hardcoded 3speak
sagarkothari88 Mar 9, 2024
2ee724c
user has low RC
sagarkothari88 Mar 9, 2024
6b5786d
fixed TS issues
sagarkothari88 Mar 11, 2024
21e3007
fixed docker mongo
sagarkothari88 Mar 11, 2024
5fd4928
addressed PR comments
sagarkothari88 Mar 26, 2024
98db453
addressed PR comments
sagarkothari88 Mar 26, 2024
d316632
Merge branches 'feature/validate-file-type' and 'main' of https://git…
sagarkothari88 Mar 26, 2024
0bc1e71
removed required
sagarkothari88 Apr 3, 2024
725eaab
Merge branches 'feature/validate-file-type' and 'main' of https://git…
sagarkothari88 Apr 3, 2024
1558ed9
resolved logical merge conflicts
sagarkothari88 Apr 3, 2024
67ae847
create upload API requires nothing. separate api is there for it.
sagarkothari88 Apr 4, 2024
c8690cb
video upload & video update is working
sagarkothari88 Apr 4, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
51 changes: 40 additions & 11 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 4 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -33,14 +33,15 @@
"diskusage": "^1.2.0",
"dlv": "^1.1.3",
"dotenv": "^16.0.3",
"fluent-ffmpeg": "^2.1.2",
"form-data": "^4.0.0",
"graphql": "^16.6.0",
"graphql-yoga": "^3.7.0",
"it-pushable": "^3.1.2",
"key-did-provider-ed25519": "^3.0.0",
"key-did-resolver": "^3.0.0",
"mailgun.js": "^10.0.0",
"kubo-rpc-client": "^3.0.2",
"mailgun.js": "^10.0.0",
"minio": "^7.1.1",
"mocha": "^10.2.0",
"moment": "^2.29.4",
Expand All @@ -58,7 +59,7 @@
"rxjs": "^7.8.0",
"slug": "^8.2.2",
"ulid": "^2.3.0",
"uuid": "^9.0.0",
"uuid": "^9.0.1",
"winston": "^3.8.2",
"ws": "^8.14.2",
"xor-distance": "^2.0.0"
Expand All @@ -68,6 +69,7 @@
"@babel/preset-typescript": "^7.23.3",
"@nestjs/passport": "^9.0.1",
"@nestjs/testing": "^9.3.2",
"@types/fluent-ffmpeg": "^2.1.24",
"@types/graphql": "^14.5.0",
"@types/jest": "^27.0.5",
"@types/multer": "^1.4.7",
Expand Down
2 changes: 1 addition & 1 deletion runTusd.sh
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@

tusd -upload-dir=/data/ -base-path="/files" -host="127.0.0.1" -port="1080" -behind-proxy --hooks-enabled-events pre-create,post-create,post-finish,post-finish,post-terminate,post-receive -max-size 5000000000 -hooks-http http://localhost:4569/api/v1/tus-callback
tusd -upload-dir=./data/ -base-path="/files" -host="127.0.0.1" -port="1080" -behind-proxy --hooks-enabled-events pre-create,post-create,post-finish,post-finish,post-terminate,post-receive -max-size 5000000000 -hooks-http http://localhost:4569/api/v1/upload/tus-callback
126 changes: 83 additions & 43 deletions src/repositories/hive/hive.repository.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,42 +3,69 @@ import hiveJsPackage from '@hiveio/hive-js';
import { AuthorPerm, OperationsArray } from "./types";
import { Client, ExtendedAccount, Operation, PrivateKey, PublicKey, Signature } from '@hiveio/dhive';
import crypto from 'crypto'
import 'dotenv/config'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, clever; I'd been using Nest's ConfigService, but its awesome if this works! We should aim to have a unified approach across the codebase though


hiveJsPackage.api.setOptions({
useAppbaseApi: true,
rebranded_api: true,
url: `https://hive-api.web3telekom.xyz`
});
hiveJsPackage.config.set('rebranded_api','true');
url: `https://hive-api.web3telekom.xyz`,
})
hiveJsPackage.config.set('rebranded_api', 'true')

@Injectable()
export class HiveRepository {
readonly #logger: Logger;
readonly #hiveJs = hiveJsPackage;
readonly #hive: Client = new Client(process.env.HIVE_HOST?.split(',') || ["https://anyx.io", "https://hived.privex.io", "https://rpc.ausbit.dev", "https://techcoderx.com", "https://api.openhive.network", "https://api.hive.blog", "https://api.c0ff33a.uk"]);
readonly #logger: Logger
readonly #hiveJs = hiveJsPackage
readonly #hive: Client = new Client(
process.env.HIVE_HOST?.split(',') || [
'https://anyx.io',
'https://hived.privex.io',
'https://rpc.ausbit.dev',
'https://techcoderx.com',
'https://api.openhive.network',
'https://api.hive.blog',
'https://api.c0ff33a.uk',
],
)

constructor() {}

async broadcastOperations(operations: OperationsArray) {
return await this.#hiveJs.broadcast.sendAsync({
operations
}, {
posting: process.env.DELEGATED_ACCOUNT_POSTING
}).catch((e: any) => {
this.#logger.error(`Error publishing operations to chain!`, operations, e)
return e;
});
return await this.#hiveJs.broadcast
.sendAsync(
{
operations,
},
{
posting: process.env.DELEGATED_ACCOUNT_POSTING,
},
)
.catch((e: any) => {
this.#logger.error(`Error publishing operations to chain!`, operations, e)
return e
})
}

async hivePostExists({ author, permlink }: AuthorPerm) {
try {
const content = await this.#hiveJs.api.getContent(author, permlink);

const content = await this.#hiveJs.api.getContent(author, permlink)
// Check if the content is an object and has a body. This implicitly checks for non-empty strings.
return typeof content === "object" && !!content.body;
return typeof content === 'object' && !!content.body
} catch (e) {
this.#logger.error('Error checking Hive post existence:', e)
return false
}
}

async hasEnoughRC({author}: {author: string;}): Promise<Boolean> {
try {
const rc = await this.#hive.rc.findRCAccounts([author]) as any[];
const rcInBillion = rc[0].rc_manabar.current_mana / 1_000_000_000;
console.log(`Resource Credits for ${author}:`, rcInBillion);
return rcInBillion > 6;
} catch (e) {
this.#logger.error("Error checking Steem post existence:", e);
return false;
this.#logger.error('Error checking Hive post existence:', e)
return false
}
}

Expand All @@ -51,35 +78,45 @@ export class HiveRepository {
}

async getAccount(author: string) {
const [hiveAccount] = await this.#hive.database.getAccounts([author]);
return hiveAccount;
const [hiveAccount] = await this.#hive.database.getAccounts([author])
return hiveAccount
}

async createAccountWithAuthority(newAccountname, authorityAccountname, options?: {
posting_auths?: string[]
active_auths?: string[]
}) {
async createAccountWithAuthority(
newAccountname,
authorityAccountname,
options?: {
posting_auths?: string[]
active_auths?: string[]
},
) {
const owner = {
weight_threshold: 1,
account_auths: [[authorityAccountname, 1]],
key_auths: [],
}
const active = {
weight_threshold: 1,
account_auths: [[authorityAccountname, 1], ...(options?.active_auths || []).map(e => {
return [e, 1]
})],
account_auths: [
[authorityAccountname, 1],
...(options?.active_auths || []).map((e) => {
return [e, 1]
}),
],
key_auths: [],
}
const posting = {
weight_threshold: 1,
account_auths: [[authorityAccountname, 1], ...(options?.posting_auths || []).map(e => {
return [e, 1]
})],
account_auths: [
[authorityAccountname, 1],
...(options?.posting_auths || []).map((e) => {
return [e, 1]
}),
],
key_auths: [],
}
const memo_key = 'STM7C9FCSZ6ntNsrwkU5MCvAB7TV44bUF8J4pwWLWpGY5Z7Ba7Q6e'

const accountData = {
creator: authorityAccountname,
new_account_name: newAccountname,
Expand All @@ -98,18 +135,17 @@ export class HiveRepository {
}),
extensions: [],
}

const operations: Operation[] = [['create_claimed_account', accountData]]

return await this.#hive.broadcast.sendOperations(
operations,
PrivateKey.fromString(process.env.ACCOUNT_CREATOR_ACTIVE), // check this
PrivateKey.fromString(process.env.ACCOUNT_CREATOR_ACTIVE), // check this
)
}

verifyHiveMessage(message: Buffer, signature: string, account: ExtendedAccount): boolean {
for (let auth of account.posting.key_auths) {

const sigValidity = PublicKey.fromString(auth[0].toString()).verify(
Buffer.from(message),
Signature.fromBuffer(Buffer.from(signature, 'hex')),
Expand Down Expand Up @@ -137,17 +173,21 @@ export class HiveRepository {
}

async decodeMessageAndGetPublicKeys(memo: string) {
const decoded = this.#hiveJs.memo.decode(process.env.DELEGATED_ACCOUNT_POSTING, memo);
const message = JSON.parse(decoded.substr(1));
const decoded = this.#hiveJs.memo.decode(process.env.DELEGATED_ACCOUNT_POSTING, memo)
const message = JSON.parse(decoded.substr(1))

return message;
return message
}

async getPublicKeys(memo: string) {
return this.#hiveJs.memo.getPubKeys(memo);
return this.#hiveJs.memo.getPubKeys(memo)
}

async comment(author: string, content: string, comment_options: { parent_author: string; parent_permlink: string; }) {
async comment(
author: string,
content: string,
comment_options: { parent_author: string; parent_permlink: string },
) {
return await this.#hive.broadcast.comment(
{
parent_author: comment_options.parent_author || '',
Expand All @@ -164,11 +204,11 @@ export class HiveRepository {
)
}

async verifyPostingAuth(account: any) {
verifyPostingAuth(account: any): Boolean {
let doWe = false
if (Array.isArray(account.posting.account_auths)) {
account.posting.account_auths.forEach(function (item) {
if (item[0] === 'threespeak') {
if (item[0] === process.env.VOTER_ACCOUNT) {
doWe = true
}
})
Expand Down
2 changes: 2 additions & 0 deletions src/repositories/upload/dto/upload.dto.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
export class UploadDto {
readonly upload_id: string;
readonly video_id: string;
readonly expires?: Date;
readonly file_name?: string;
Expand All @@ -7,4 +8,5 @@ export class UploadDto {
readonly cid?: string;
readonly type: 'video' | 'thumbnail' | 'other';
readonly created_by: string;
readonly immediatePublish: boolean;
}
6 changes: 6 additions & 0 deletions src/repositories/upload/schemas/upload.schema.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ export type UploadDocument = HydratedDocument<Upload>;

@Schema()
export class Upload {
@Prop({ type: String, required: true, default: () => ulid() })
upload_id: string;

@Prop({ type: String, required: true, default: () => ulid() })
video_id: string;

Expand All @@ -18,6 +21,9 @@ export class Upload {
@Prop({ type: String, required: false })
file_path: string;

@Prop({ type: Boolean, required: false, default: false })
immediatePublish: boolean;

@Prop({ type: String, required: true, enum: ['pending', 'done', 'error'] })
ipfs_status: 'pending' | 'done' | 'error';

Expand Down