mirrors
/
AdGuardHome
mirror of https://github.com/AdguardTeam/AdGuardHome.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
AdGuardHome/scripts
History
Stanislav Chzhen c47509fabc Pull request 1928: 1453-stats-tests 
Updates #1453.

Squashed commit of the following:

commit f08f68ef5493dad03d3eb120d886f2df1af28be6
Merge: b70b088af 54aee2272
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Tue Aug 8 19:04:06 2023 +0300

    Merge branch 'master' into 1453-stats-tests

commit b70b088af0fdc7d6d048d688160048bad1fceb12
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Thu Aug 3 19:32:04 2023 +0300

    stats: imp code

commit c341012ba61894c255c1868624be1cac0d26a6fa
Merge: a2ac8c34e 5eb3cd0f9
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Thu Aug 3 13:36:24 2023 +0300

    Merge branch 'master' into 1453-stats-tests

commit a2ac8c34ee32606ca5e259c3e2a47db0dd5858de
Author: Ildar Kamalov <ik@adguard.com>
Date:   Thu Aug 3 13:25:12 2023 +0300

    client: add top upstreams and average processing time tables

commit 11118947f9bf945be0b056f8475cf3b848c6e66e
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Tue Aug 1 17:24:57 2023 +0300

    stats: imp docs

commit 904cf81d02a1f327b9647fa7ad9e181cfabb68a4
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Mon Jul 31 17:34:06 2023 +0300

    stats: imp code

commit 34f0c96dd5865d1470385322a88842dd0b3d996d
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Mon Jul 31 15:43:46 2023 +0300

    all: imp docs

commit 2cb2d0d8bef3580f64bc25c414fe9b5ea6b9f997
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Fri Jul 28 17:24:31 2023 +0300

    all: imp code

commit 5251a899fecc21e50a0ba06042f96f5b404e196a
Merge: b6c2b12d4 300821a7f
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Thu Jul 27 20:34:39 2023 +0300

    Merge branch 'master' into 1453-stats-tests

commit b6c2b12d4425012efd73549c3a426735f3a677cd
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Thu Jul 27 20:32:18 2023 +0300

    stats: imp code

commit 5546b82a78326f9cc6d8c87df5083f8fc66a0178
Merge: 8a3d6b1b4 5f8fa006c
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Thu Jul 27 14:24:01 2023 +0300

    Merge branch 'master' into 1453-stats-tests

commit 8a3d6b1b49ce189f95adfa7406a34108e885e676
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Thu Jul 27 14:17:47 2023 +0300

    all: imp code

commit 2a48001e275e3cdcf70e13e1c9cebd4e502f3259
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Tue Jul 25 18:27:20 2023 +0300

    all: imp docs

commit 3dd21890175af32a3368378f7e013383f6d040ec
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Tue Jul 25 16:00:39 2023 +0300

    all: imp naming

commit 6124456fc3149b71f6bd58d35ecf24eb6cf40d5d
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Thu Jul 20 16:15:56 2023 +0300

    all: add upstreams avg processing time

commit 187ad0c77a81c9fd95c24e23141355db2e83e50d
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Tue Jul 18 16:42:19 2023 +0300

    all: add top upstreams
 		2023-08-09 14:33:52 +03:00
..
blocked-services Pull request: 3972-hostlists-services 2022-10-27 15:46:25 +03:00
companiesdb Pull request: 5471-new-trackers-db 2023-02-14 13:52:22 +03:00
hooks Pull request 1804: newline-lint 2023-04-06 14:21:46 +03:00
make Pull request 1928: 1453-stats-tests 2023-08-09 14:33:52 +03:00
querylog Pull request 1804: newline-lint 2023-04-06 14:21:46 +03:00
snap Pull request 1884: AG-23334-split-snap 2023-06-21 17:14:10 +03:00
translations Pull request 1934: AG-24191-blocker-languages 2023-07-20 19:52:14 +03:00
vetted-filters Pull request 1937: imp-filter-upd 2023-07-25 17:47:24 +03:00
README.md Pull request 1934: AG-24191-blocker-languages 2023-07-20 19:52:14 +03:00
install.sh Pull request: 5431 Privileged install 2023-02-16 18:57:44 +03:00

README.md

AdGuard Home Scripts

hooks/: Git Hooks

Usage

Run make init from the project root.

querylog/: Query Log Helpers

Usage

  • npm install: install dependencies. Run this first.
  • npm run anonymize <source> <dst>: read the query log from the <source> and write anonymized version to <dst>.

make/: Makefile scripts

The release channels are: development (the default), edge, beta, and release. If verbosity levels aren't documented here, there are only two: 0, don't print anything, and 1, be verbose.

build-docker.sh: Build a multi-architecture Docker image

Required environment:

  • CHANNEL: release channel, see above.

  • COMMIT: current Git revision.

  • DIST_DIR: the directory where a release has previously been built.

  • VERSION: release version.

Optional environment:

  • DOCKER_IMAGE_NAME: the name of the resulting Docker container. By default it's adguardhome-dev.

  • DOCKER_OUTPUT: the --output parameters. By default they are type=image,name=${DOCKER_IMAGE_NAME},push=false.

  • SUDO: allow users to use sudo or doas with docker. By default none is used.

build-release.sh: Build a release for all platforms

Required environment:

  • CHANNEL: release channel, see above.

  • GPG_KEY and GPG_KEY_PASSPHRASE: data for gpg. Only required if SIGN is 1.

Optional environment:

  • ARCH and OS: space-separated list of architectures and operating systems for which to build a release. For example, to build only for 64-bit ARM and AMD on Linux and Darwin:

    make ARCH='amd64 arm64' OS='darwin linux' … build-release

    The default value is '', which means build everything.

  • DIST_DIR: the directory to build a release into. The default value is dist.

  • GO: set an alternative name for the Go compiler.

  • SIGN: 0 to not sign the resulting packages, 1 to sign. The default value is 1.

  • VERBOSE: 1 to be verbose, 2 to also print environment. This script calls go-build.sh with the verbosity level one level lower, so to get verbosity level 2 in go-build.sh, set this to 3 when calling build-release.sh.

  • VERSION: release version. Will be set by version.sh if it is unset or if it has the default Makefile value of v0.0.0.

clean.sh: Cleanup

Optional environment:

  • GO: set an alternative name for the Go compiler.

Required environment:

  • DIST_DIR: the directory where a release has previously been built.

go-bench.sh: Run backend benchmarks

Optional environment:

  • GO: set an alternative name for the Go compiler.

  • TIMEOUT_FLAGS: set timeout flags for tests. The default value is --timeout=30s.

  • VERBOSE: verbosity level. 1 shows every command that is run and every Go package that is processed. 2 also shows subcommands and environment. The default value is 0, don't be verbose.

go-build.sh: Build the backend

Optional environment:

  • GOARM: ARM processor options for the Go compiler.

  • GOMIPS: ARM processor options for the Go compiler.

  • GO: set an alternative name for the Go compiler.

  • OUT: output binary name.

  • PARALLELISM: set the maximum number of concurrently run build commands (that is, compiler, linker, etc.).

  • SOURCE_DATE_EPOCH: the standardized environment variable for the Unix epoch time of the latest commit in the repository. If set, overrides the default obtained from Git. Useful for reproducible builds.

  • VERBOSE: verbosity level. 1 shows every command that is run and every Go package that is processed. 2 also shows subcommands and environment. The default value is 0, don't be verbose.

  • VERSION: release version. Will be set by version.sh if it is unset or if it has the default Makefile value of v0.0.0.

Required environment:

  • CHANNEL: release channel, see above.

go-deps.sh: Install backend dependencies

Optional environment:

  • GO: set an alternative name for the Go compiler.

  • VERBOSE: verbosity level. 1 shows every command that is run and every Go package that is processed. 2 also shows subcommands and environment. The default value is 0, don't be verbose.

go-fuzz.sh: Run backend fuzz tests

Optional environment:

  • GO: set an alternative name for the Go compiler.

  • FUZZTIME_FLAGS: set fuss flags for tests. The default value is --fuzztime=20s.

  • TIMEOUT_FLAGS: set timeout flags for tests. The default value is --timeout=30s.

  • VERBOSE: verbosity level. 1 shows every command that is run and every Go package that is processed. 2 also shows subcommands and environment. The default value is 0, don't be verbose.

go-lint.sh: Run backend static analyzers

Don't forget to run make go-tools once first!

Optional environment:

  • EXIT_ON_ERROR: if set to 0, don't exit the script after the first encountered error. The default value is 1.

  • GO: set an alternative name for the Go compiler.

  • VERBOSE: verbosity level. 1 shows every command that is run. 2 also shows subcommands. The default value is 0, don't be verbose.

go-test.sh: Run backend tests

Optional environment:

  • GO: set an alternative name for the Go compiler.

  • RACE: set to 0 to not use the Go race detector. The default value is 1, use the race detector.

  • TIMEOUT_FLAGS: set timeout flags for tests. The default value is --timeout=30s.

  • VERBOSE: verbosity level. 1 shows every command that is run and every Go package that is processed. 2 also shows subcommands. The default value is 0, don't be verbose.

go-tools.sh: Install backend tooling

Installs the Go static analysis and other tools into ${PWD}/bin. Either add ${PWD}/bin to your $PATH before all other entries, or use the commands directly, or use the commands through make (for example, make go-lint).

Optional environment:

  • GO: set an alternative name for the Go compiler.

version.sh: Generate And Print The Current Version

Required environment:

  • CHANNEL: release channel, see above.

snap/: Snapcraft scripts

build.sh

Builds the Snapcraft packages from the binaries created by download.sh.

download.sh

Downloads the binaries to pack them into Snapcraft packages.

Required environment:

  • CHANNEL: release channel, see above.

upload.sh

Uploads the Snapcraft packages created by build.sh.

Required environment:

  • SNAPCRAFT_CHANNEL: Snapcraft release channel: edge, beta, or candidate.

  • SNAPCRAFT_STORE_CREDENTIALS: Credentials for Snapcraft store.

Optional environment:

  • SNAPCRAFT_CMD: Overrides the Snapcraft command. Default: snapcraft.

translations/: Twosky Integration Script

Usage

  • go run ./scripts/translations help: print usage.

  • go run ./scripts/translations download [-n <count>]: download and save all translations. n is optional flag where count is a number of concurrent downloads.

  • go run ./scripts/translations upload: upload the base en locale.

  • go run ./scripts/translations summary: show the current locales summary.

  • go run ./scripts/translations unused: show the list of unused strings.

  • go run ./scripts/translations auto-add: add locales with additions to the git and restore locales with deletions.

After the download you'll find the output locales in the client/src/__locales/ directory.

Optional environment:

  • DOWNLOAD_LANGUAGES: set a list of specific languages to download. For example ar be bg. If it set to blocker then script will download only those languages, which need to be fully translated (de en es fr it ja ko pt-br pt-pt ru zh-cn zh-tw).

  • UPLOAD_LANGUAGE: set an alternative language for upload.

  • TWOSKY_URI: set an alternative URL for download or upload.

  • TWOSKY_PROJECT_ID: set an alternative project ID for download or upload.

companiesdb/: Whotracks.me Database Converter

A simple script that downloads and updates the companies DB in the client code from the repo.

Usage

sh ./scripts/companiesdb/download.sh

blocked-services/: Blocked Services Updater

A simple script that downloads and updates the blocked services index from AdGuard's Hostlists Registry.

Optional environment:

  • URL: the URL of the index file. By default it's https://adguardteam.github.io/HostlistsRegistry/assets/services.json.

Usage

go run ./scripts/blocked-services/main.go

vetted-filters/: Vetted Filters Updater

Similar to the one above, a script that downloads and updates the vetted filtering list data from AdGuard's Hostlists Registry.

Optional environment:

  • URL: the URL of the index file. By default it's https://adguardteam.github.io/HostlistsRegistry/assets/filters.json.

Usage

go run ./scripts/vetted-filters/main.go