Commit Graph

120 Commits

Author SHA1 Message Date
Darius Kazemi f2a6028834 Merge tag 'v4.1.0rc1' into hometown-dev 2023-01-23 16:46:13 -08:00
Nick Schonning 06b68490d1
Enable eslint:recommended ruleset (#22433)
* Enable ESLint recommended ruleset

* Disable failing ESLint recommended rules

* Remove rules shadowed by eslint:recommended
2022-12-19 00:51:37 +09:00
Darius Kazemi 5732db7348 Merge tag 'v4.0.2' into hometown-4.0-merge 2022-12-04 12:01:49 -08:00
Darius Kazemi 840688318f Merge tag 'v4.0.0' into hometown-4.0-merge 2022-11-16 20:54:49 -08:00
Claire 8c3c6dcae9
Fix filters from other users being used in the streaming service (#20719) 2022-11-15 02:09:58 +01:00
Claire cd5e98dbdb
Fix public/local timeline posts not being properly filtered (#20567)
* Fix streaming server using wrong property name for matching filters

Late in the PR, the `filter_results` property has been renamed to `filtered`,
but the change has not been reflected in the streaming server code.

* Fix filter_action attribute being an integer instead of a string
2022-11-13 20:59:49 +01:00
Eugen Rochko e7aa2be828
Change how hashtags are normalized (#18795)
* Change how hashtags are normalized

* Fix tests
2022-07-13 15:03:28 +02:00
Claire 02851848e9
Revamp post filtering system (#18058)
* Add model for custom filter keywords

* Use CustomFilterKeyword internally

Does not change the API

* Fix /filters/edit and /filters/new

* Add migration tests

* Remove whole_word column from custom_filters (covered by custom_filter_keywords)

* Redesign /filters

Instead of a list, present a card that displays more information and handles
multiple keywords per filter.

* Redesign /filters/new and /filters/edit to add and remove keywords

This adds a new gem dependency: cocoon, as well as a npm dependency:
cocoon-js-vanilla. Those are used to easily populate and remove form fields
from the user interface when manipulating multiple keyword filters at once.

* Add /api/v2/filters to edit filter with multiple keywords

Entities:
- `Filter`: `id`, `title`, `filter_action` (either `hide` or `warn`), `context`
  `keywords`
- `FilterKeyword`: `id`, `keyword`, `whole_word`

API endpoits:
- `GET /api/v2/filters` to list filters (including keywords)
- `POST /api/v2/filters` to create a new filter
  `keywords_attributes` can also be passed to create keywords in one request
- `GET /api/v2/filters/:id` to read a particular filter
- `PUT /api/v2/filters/:id` to update a new filter
  `keywords_attributes` can also be passed to edit, delete or add keywords in
   one request
- `DELETE /api/v2/filters/:id` to delete a particular filter
- `GET /api/v2/filters/:id/keywords` to list keywords for a filter
- `POST /api/v2/filters/:filter_id/keywords/:id` to add a new keyword to a
   filter
- `GET /api/v2/filter_keywords/:id` to read a particular keyword
- `PUT /api/v2/filter_keywords/:id` to edit a particular keyword
- `DELETE /api/v2/filter_keywords/:id` to delete a particular keyword

* Change from `irreversible` boolean to `action` enum

* Remove irrelevent `irreversible_must_be_within_context` check

* Fix /filters/new and /filters/edit with update for filter_action

* Fix Rubocop/Codeclimate complaining about task names

* Refactor FeedManager#phrase_filtered?

This moves regexp building and filter caching to the `CustomFilter` class.

This does not change the functional behavior yet, but this changes how the
cache is built, doing per-custom_filter regexps so that filters can be matched
independently, while still offering caching.

* Perform server-side filtering and output result in REST API

* Fix numerous filters_changed events being sent when editing multiple keywords at once

* Add some tests

* Use the new API in the WebUI

- use client-side logic for filters we have fetched rules for.
  This is so that filter changes can be retroactively applied without
  reloading the UI.
- use server-side logic for filters we haven't fetched rules for yet
  (e.g. network error, or initial timeline loading)

* Minor optimizations and refactoring

* Perform server-side filtering on the streaming server

* Change the wording of filter action labels

* Fix issues pointed out by linter

* Change design of “Show anyway” link in accordence to review comments

* Drop “irreversible” filtering behavior

* Move /api/v2/filter_keywords to /api/v1/filters/keywords

* Rename `filter_results` attribute to `filtered`

* Rename REST::LegacyFilterSerializer to REST::V1::FilterSerializer

* Fix systemChannelId value in streaming server

* Simplify code by removing client-side filtering code

The simplifcation comes at a cost though: filters aren't retroactively
applied anymore.
2022-06-28 09:42:13 +02:00
Darius Kazemi 2c5862ede0 Merge tag 'v3.5.2' into hometown-dev-3.5.2 2022-05-05 21:08:15 -07:00
Yamagishi Kazutoshi 5781d1db84
Fix parsing `TRUSTED_PROXY_IP` (#18051) 2022-04-19 08:11:58 +01:00
Darius Kazemi c7e5c4a8a6 Merge tag 'v3.5.0' into hometown-dev-3.5.0 2022-04-01 14:53:35 -07:00
Claire f29458da1d
Fix streaming server sometimes silently dropping subscriptions (#17841) 2022-03-21 19:08:29 +01:00
Claire 6ea80ba2a2
Change streaming server error messages when failing to parse client input (#17559)
Fixes #17541

- prefix JSON parsing error message by “Error parsing message from …”
- output user id if a user is logged in, IP address otherwise
- reduce log level from error to warning when a user is logged in, and to silly
  otherwise
2022-02-16 14:37:26 +01:00
Claire bc066d8936
Fix timeline streaming stopping for multiple sessions instead of one (#17259)
* Fix timeline streaming stopping for multiple sessions instead of one

Fixes #17256.

In updating the code for a newer version of node-redis, #17183 also broke
redis subscription management when multiple streaming clients subscribe to the
same channel.

This commit restores the redis subscription management code.

* Let node-redis actually handle the subscriptions
2022-01-07 19:50:12 +01:00
Lerk 4d1eaf3e6e
Finish update of node-redis (#17183)
* fix streaming redis client

* use console.error instead of console.log

* follow node-redis migration guide

https://github.com/redis/node-redis/blob/master/docs/v3-to-v4.md

* fix config options for node-redis

* keep indentation

* Update streaming/index.js

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2021-12-25 22:55:06 +01:00
Sasha Sorokin 6c88ebfd4b
fix(streaming): req.scopes can be nullable (#16823)
When checking for required OAuth scopes, an unexpected error could
happen due to missing (null-y) req.scopes. This commit fixes that by
checking if req.scopes are present before checking if any required
scopes are present, otherwise it skips that straight to rejection.
2021-10-13 05:02:55 +02:00
Eugen Rochko a0d4129893
Refactor notifications to go through a separate stream in streaming API (#16765)
Eliminate need to have custom notifications filtering logic in the
streaming API code by publishing notifications into a separate stream
and then simply using the multi-stream capability to subscribe to that
stream when necessary
2021-09-26 13:23:28 +02:00
Darius Kazemi f57cda3855 Merge tag 'v3.4.0' into hometown-dev 2021-05-17 13:48:27 -07:00
Eugen Rochko c5c46dd6ee
Fix "cb is not a function" error in streaming API server (#16134)
Third argument of `ping` is the callback

Regression from #15932
2021-05-02 14:30:26 +02:00
Eugen Rochko aafe65a142
Change log level of worker start/end to warn in streaming API (#16110) 2021-05-01 23:19:18 +02:00
Claire 49814d5799
Switch from deprecated ClusterWS/cws to ws package (#15932)
* Switch from deprecated ClusterWS/cws to ws package

Fixes #15184

Co-authored-by: Edho Arief <me@nanaya.pro>

* Make bufferutil and utf-8-validate optional dependencies

Co-authored-by: Edho Arief <me@nanaya.pro>
2021-03-24 09:37:41 +01:00
Renato "Lond" Cerqueira cb085b4c44 Merge tag 'v3.3.0' into instance_only_statuses 2020-12-27 11:19:14 +01:00
abcang a2da02626e
Fixed ESLint error (#15214)
* eslint --fix

* fix consistent-return

* fix promise/catch-or-return

* ignore import rule
2020-11-23 17:35:14 +01:00
Eugen Rochko aa10200e58
Fix streaming API allowing connections to persist after access token invalidation (#15111)
Fix #14816
2020-11-12 23:05:24 +01:00
fuyu e39d97f700
Fix crash streaming process when receive invalid json (#14859) 2020-09-22 15:30:41 +02:00
Eugen Rochko 01647b8acb
Fix destructuring error when unsubscribing without subscribing (#14566) 2020-08-12 15:36:07 +02:00
Eugen Rochko ef057584fd
Add support for managing multiple stream subscriptions in a single connection (#14524) 2020-08-11 18:24:59 +02:00
Renato "Lond" Cerqueira 8126ac030e Merge tag 'v3.2.0' into instance_only_statuses 2020-07-27 22:21:35 +02:00
Renato "Lond" Cerqueira 06b6353483 Merge tag 'v3.1.5' into instance_only_statuses 2020-07-15 20:13:00 +02:00
ThibG ddcdddd6fc
Change streaming server to treat blank redis password as password-less auth (#14135)
Fixes #14131

Our `mastodon:setup` task defaults to a blank password rather than the
absence of password, but some versions of Redis reject blank password
authentication when authentication is possible without a password.

The Ruby code only uses the Redis password when it's not blank, so
do the same for the node.js part.
2020-06-24 22:25:23 +02:00
Eugen Rochko 5d8398c8b8
Add E2EE API (#13820) 2020-06-02 19:24:53 +02:00
Takeshi Umeda 26b08a3c54
Add remote only to public timeline (#13504)
* Add remote only to public timeline

* Fix code style
2020-05-10 10:36:18 +02:00
Renato "Lond" Cerqueira e0f3a4583c Merge tag 'v3.1.1' into instance_only_statuses 2020-02-21 14:21:59 +01:00
Ben Lubar 0dfba0884e minor server-sent events fixes (#12945)
* Send output on the server-sent events stream immediately so the client sees that it was successfully opened even if it doesn't have any messages.

Fix transparent SSE streaming for the public:local and hashtag:local stream types.

* Tell caches to never store server-sent events.
2020-01-24 20:51:33 +01:00
Renato "Lond" Cerqueira a42a98bc67 Merge tag 'v3.0.1' into instance_only_statuses 2019-10-21 09:14:53 +02:00
Renato "Lond" Cerqueira fbaaf24be8 Merge tag 'v2.9.3' into instance_only_statuses 2019-08-10 09:20:44 +02:00
Eugen Rochko 291d868773 Change default interface of web and streaming from 0.0.0.0 to 127.0.0.1 (#11302) 2019-08-09 21:32:48 +02:00
Eugen Rochko 24552b5160
Add whitelist mode (#11291) 2019-07-30 11:10:46 +02:00
Eugen Rochko e7353c47db
Change default interface of web and streaming from 0.0.0.0 to 127.0.0.1 (#11302) 2019-07-15 05:56:35 +02:00
Renato "Lond" Cerqueira f0a512c7ca Merge tag 'v2.9.2' into instance_only_statuses 2019-06-25 19:44:59 +02:00
Renato "Lond" Cerqueira 1e5b17b88a Merge tag 'v2.8.4' into instance_only_statuses 2019-06-03 22:06:36 +02:00
ThibG aa80292170 Improve streaming server security (#10818)
* Check OAuth token scopes in the streaming API

* Use Sec-WebSocket-Protocol instead of query string to pass WebSocket token

Inspired by https://github.com/kubevirt/kubevirt/issues/1242
2019-05-24 15:23:38 +02:00
ThibG d63c3c0cef Improve streaming server security (#10818)
* Check OAuth token scopes in the streaming API

* Use Sec-WebSocket-Protocol instead of query string to pass WebSocket token

Inspired by https://github.com/kubevirt/kubevirt/issues/1242
2019-05-24 15:21:42 +02:00
abcang ca6c93a2f5 Migrate from uws to cws (#10805) 2019-05-22 18:19:16 +02:00
Renato "Lond" Cerqueira 546581d2ba Merge tag 'v2.8.0' into instance_only_statuses 2019-04-13 23:55:06 +02:00
Eugen Rochko 13a7f05030
Fix streaming API always attempting to use SSL with Postgres (#10231)
Fix #10223
2019-03-11 00:51:23 +01:00
Eugen Rochko dbeab5a036
Fix SSL configuration regression in streaming API (#10225)
* Fix SSL configuration regression in streaming API

Fix #10223

* Fix code style issues and integrate #10219

* Fix dumb thing
2019-03-10 16:00:54 +01:00
Sascha 42e733681a config: add DB_SSLMODE for managed/remote PG (#10210)
* config: add DB_SSLMODE for managed/remote PG

* streaming: set PG sslmode, defaults to prefer
2019-03-08 14:36:28 +01:00
Thibaut Girka 237952c10d Only stream local-only toots to logged-in users 2018-12-07 15:32:15 +01:00
Gomasy 8d70d3de38 Fix crash when using UNIX socket (#9036) 2018-10-21 16:41:33 +02:00