Commit Graph

74 Commits

Author SHA1 Message Date
nachtjasmin b200e4ad1e
Add missing transcoder options back 2023-12-27 01:14:46 +01:00
nachtjasmin cc42aa5c08
Merge tag 'v4.2.1' into lets-bump-hometown-to-mastodon-4.2 2023-11-18 17:00:16 +01:00
nachtjasmin a31fc74580
Merge tag 'v4.2.0-rc2' into lets-bump-hometown-to-mastodon-4.2
Conflict in lib/paperclip/transcoder.rb, using upstream version.
2023-11-18 16:59:27 +01:00
nachtjasmin d6f67e3d4e
Merge tag 'v4.2.0-beta3' into lets-bump-hometown-to-mastodon-4.2 2023-11-18 01:25:59 +01:00
nachtjasmin 9a629eba8a
Merge tag 'v4.2.0-beta1' into lets-bump-hometown-to-mastodon-4.2
- cli: followed upstream
- version.rb: followed upstream, since we can use environment variables for the suffix now
- lib/paperclip: chose their spoof detector
- lib/sanitize: merged h1-h6 into supported elements, allowed translated attribute
- config/environments: follow upstream
- config/initializers: follow upstream
- config/application.rb: follow upstream
- config/locales: translations with %{title} prefix were replaced with hardcoded "Mastodon:" prefixes, should be fixed afterwards
  it's inconsistent anyway right now
- config/settings: removed settings that were removed in upstream
- config/routes: followed upstream, due to API restructurings. Is there some hometown-specific API stuff that might be missing now?
- spec/: followed upstream, might have lost hometown-specific tests, but I haven't found any on a quick check

- .ruby-version, Gemfile, etc.: upstream
- .github/workflows: upstream

- about: followed upstream, therefore the static homepage is gone :/
- credentials: moved federation into the settings_attributes
- lists: follow upstream, `:is_exclusive` -> `:exclusive`
- statuses: keep local only

- account_statuses_filter: still hide local only posts for anonymous users
- activitypub/activity/create:
  - keep activity_pub_type in params
  - text: use hometown's way for determining the content
  - spoiler: use hometown's mechanism
- feed_manager: use upstream exclusive list mechanism
- plain_text_formatter: use upstream way with html decoding, as I'm not sure whether we still have the Nokogiri library(?) available
    problem: might remove tags that we want to keep?
- text_formatter: follow upstream

- account: use upstream MENTION_RE expression
- backup: follow upstream for permission validation
- list: follow upstream, is_exclusive -> exclusive !! WE MIGHT NEED A MIGRATION!
- status: moved set_locality hook to the others
- user: delegates for settings (federation, autoplay, etc.) were removed upstream, follow them
- webhook: follow upstream

- initial_state_serializer: keep max_toot_chars
- list_serializer: follow rename of is_exclusive -> exclusive

Use upstream version, since the translation API got upgraded to v2.

Use upstream version of vote_validator.

- admin/webhooks/_form: add group for template
- settings/preferences/appearance/show: add new input for 'expand_usernames'
    check: missing translations, especially for hints
- settings/preferences/other/show: added input groups for no_rss and default_privacy
    check: missing translations, especially for hints
- settigngs/profiles: upstream removed verification banner, follow them

Followed upstream changes.

Incorporated upstream changes and put the local_only check back in the correct place.

Ignored hometown changes, take upstream version.

- actions/lists: follow exclusive list naming
- components/column_back_button: follow upstream router refactoring
- components/column_header: follow upstream router refactoring
- components/hashtag: keep hometown behaviour, add href to links
- components/media_gallery: merge alt text indicator into upstream
- components/status: merge timestamp click -> original page
- components/status_action_bar: upstream removed the share button, follow them
- components/status_content:
  - still make remote usernames => check: does the new href work?
  - make translate button always visible like upstream
  - keep hometown-specific changes for Articles and other posts
- features/header: keep header link
- features/account_gallery/components/media_item: keep link
- features/audio/index: keep no media description indicator, merge upstream styles
- features/compose/components/compose_form:
  - merge max chars logic
  - merge federation dropdown
- features/compose/components/navigation_bar: keep href to profile
- features/compose/components/poll_form: keep "is multiple" toggle
- features/compose/index: keep column header
- features/follow_requests/components/account_authorize: keep external link
- features/list_editor/components/edit_list_form: overwritten from upstream
- features/list_timeline/index: overwritten from upstream
- features/components/follow_request: keep external link
- features/components/notification: keep external link
- features/picture_in_picture/components/footer: keep external link
- features/status/components/detailed_status: keep external link
- features/ui/components/boost_modal: keep external link
- features/ui/index: merge upstream changes
- features/video/: keep no media description indicator
- containers/status_container: overwrite with upstream
- locales: best-effort merge, but I wouldn't trust it. should be normalized in some way.
2023-11-18 01:14:04 +01:00
Claire 916b5bd4ad Fix division by zero in video in bitrate computation code (#27129) 2023-10-10 13:52:41 +02:00
Jasmin 3b69a29703
Merge changes of 4.0.7..4.0.10 (#1324)
This includes all changes made in
https://github.com/mastodon/mastodon/releases/tag/v4.0.7 and
https://github.com/mastodon/mastodon/releases/tag/v4.0.8 and therefore
requires:

⚠️ Restart of all Mastodon processes, especially the Streaming
API

There was a conflict during the merge inside the `streaming/index.js`,
which was related to the following snippet:

```js
      // Only send local-only statuses to logged-in users
      if (payload.local_only && !req.accountId) {
        log.silly(req.requestId, `Message ${payload.id} filtered because it was local-only`);
        return;
      }
```

I've placed it at the same spot where it was previously, so everything
should work as before. ~~**However, I still need to test those changes,
which is what I'll be doing after submitting this PR as draft.**~~

Checked the changes against our instance, everything is working as
expected as logged-in user. Haven't checked the public timelines where
the above snippet would be relevant, as we are using `AUTHORIZED_FETCH`
and I would like to keep it enabled. ^^

If the base branch is not correct, feel free to change it.

Everything is already deployed on queer.group and working fine there.

---------

Co-authored-by: Michael Stanclift <mx@vmstan.com>
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
Co-authored-by: Emelia Smith <ThisIsMissEm@users.noreply.github.com>
Co-authored-by: Renaud Chaput <renchap@gmail.com>
Co-authored-by: Daniel M Brasil <danielmbrasil@protonmail.com>
Co-authored-by: yufushiro <62991447+yufushiro@users.noreply.github.com>
Co-authored-by: Nicolai Søborg <NicolaiSoeborg@users.noreply.github.com>
2023-09-19 19:49:27 -07:00
Eugen Rochko 0f3f9b611f
Change video bitrate to always fit within size limit (#26970) 2023-09-19 12:25:58 +02:00
Eugen Rochko 01b87a1632
Change video compression parameters (#26631) 2023-08-28 19:40:08 +02:00
yufushiro 3aac12981c
Fix unexpected audio stream transcoding when uploaded video is eligible to passthrough (#26608)
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2023-08-23 08:44:56 +02:00
Christian Schmidt f2257069b2
Fix AVIF attachments (#26264) 2023-08-01 19:34:11 +02:00
Jasmin 0728a6a709
Merge upstream security fixes of v4.0.5 (#1316)
It's already running on our instance (queer.group) and working fine.

Manually reviewed the changes, hadn't found anything that could break
hometown-specific code.
And to update our instance, I also just followed the [steps on the
release](https://github.com/mastodon/mastodon/releases/tag/v4.0.5) aka
`bundle install && yarn install` followed by a restart of all processes.

---------

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
Co-authored-by: Daniel M Brasil <danielmbrasil@protonmail.com>
Co-authored-by: Emelia Smith <ThisIsMissEm@users.noreply.github.com>
Co-authored-by: Vyr Cossont <VyrCossont@users.noreply.github.com>
Co-authored-by: Renaud Chaput <renchap@gmail.com>
2023-07-06 12:30:13 -07:00
Claire dc8f1fbd97
Merge pull request from GHSA-9928-3cp5-93fm
* Fix attachments getting processed despite failing content-type validation

* Add a restrictive ImageMagick security policy tailored for Mastodon

* Fix misdetection of MP3 files with large cover art

* Reject unprocessable audio/video files instead of keeping them unchanged
2023-07-06 15:05:05 +02:00
Nick Schonning c66250abf1
Autofix Rubocop Regex Style rules (#23690)
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2023-06-06 14:50:51 +02:00
Matt Jankowski 5e060e1f44
Fix Performance/Sum cop (#24788) 2023-05-02 16:10:40 +02:00
Nick Schonning 0cfdd1a401
Enable Rubocop Style/StringConcatenation defaults (#23792) 2023-02-22 09:54:36 +09:00
Nick Schonning af4c95100c
Autofix Rubocop Style/FormatString (#23743) 2023-02-20 07:58:33 +01:00
Nick Schonning d2dcb6c45a
Autofix Rubocop Style/UnpackFirst (#23741) 2023-02-20 06:51:43 +01:00
Nick Schonning 2177daeae9
Autofix Rubocop Style/RedundantBegin (#23703) 2023-02-19 07:09:40 +09:00
Nick Schonning a6f77aa28a
Autofix Rubocop Lint/AmbiguousOperatorPrecedence (#23681) 2023-02-18 04:30:23 +01:00
Nick Schonning 669f6d2c0a
Run rubocop formatting except line length (#23632) 2023-02-18 06:56:20 +09:00
Pierre Bourdon 36bc90e8aa
blurhash_transcoder: prevent out-of-bound reads with <8bpp images (#20388)
The Blurhash library used by Mastodon requires an input encoded as 24
bits raw RGB data. The conversion to raw RGB using Imagemagick did not
previously specify the desired bit depth. In some situations, this leads
Imagemagick to output in a pixel format using less bpp than expected.
This then manifested as segfaults of the Sidekiq process due to
out-of-bounds read, or potentially a (highly noisy) memory infoleak.

Fixes #19235.
2022-11-11 07:45:16 +01:00
Claire 9d4861b498
Remove dependency on running Redis server for db:setup (#18560) 2022-06-01 19:23:31 +02:00
Holger 39b489ba4c
fix: `s3_force_single_request` not parsed (#17922) 2022-04-01 23:56:23 +02:00
Claire 166f6e4b50
Fix some media attachments being converted with too high framerates (#17619)
Video files with variable framerates are converted to constant framerate videos
and the output framerate picked by ffmpeg is based on the original file's
container framerate (which can be different from the average framerate).

This means that an input video with variable framerate with about 30 frames per
second on average, but a maximum of 120 fps will be converted to a constant 120
fps file, which won't be processed by other Mastodon servers.

This commit changes it so that input files with VFR and a maximum framerate
above the framerate threshold are converted to VFR files with the maximum frame
rate enforced.
2022-02-22 17:11:22 +01:00
Claire 48f8658d34
Fix upload of remote media with OpenStack Swift sometimes failing (#16998)
Under certain conditions, files fetched from remotes trigger an error when
being uploaded using OpenStack Swift. This is because in some cases, the
remote server will not return a content-length, so our ResponseWithLimitAdapter
will hold a `nil` value for `#size`, which will lead to an invalid value
for the Content-Length header of the Swift API call.

This commit fixes that by taking the size from the actually-downloaded file
size rather than the upstream-provided Content-Length header value.
2021-11-16 21:36:28 +01:00
Claire 6ba8bc45cb
Add S3_FORCE_SINGLE_REQUEST env var to work around S3 compatibility issues (#16866)
Fixes #16822
2021-10-18 18:29:04 +02:00
Claire fc3ae1343d
Switch from unmaintained paperclip to kt-paperclip (#16724)
* Switch from unmaintained paperclip to kt-paperclip

* Drop some compatibility monkey-patches not required by kt-paperclip

* Drop media spoof check monkey-patching

It's broken with kt-paperclip and hopefully it won't be needed anymore

* Fix regression introduced by paperclip 6.1.0

* Do not rely on pathname to call FastImage

* Add test for ogg vorbis file with cover art

* Add audio/vorbis to the accepted content-types

This seems erroneous as this would be the content-type for a vorbis stream
without an ogg container, but that's what the `marcel` gem outputs, so…

* Restore missing for_as_default method

* Refactor Attachmentable concern and delay Paperclip's content-type spoof check

Check for content-type spoofing *after* setting the extension ourselves, this
fixes a regression with kt-paperclip's validations being more strict than
paperclip 6.0.0 and rejecting some Pleroma uploads because of unknown
extensions.

* Please CodeClimate

* Add audio/vorbis to the unreliable set

It doesn't correspond to a file format and thus has no extension associated.
2021-09-29 23:52:36 +02:00
Takeshi Umeda 818e0b314f
Fix unsupported video error message handling (#16581) 2021-08-08 15:28:57 +02:00
rinsuki 5ed5f62705
Fix animated GIF generates animated thumbnail (#16216) 2021-05-11 19:15:11 +02:00
Claire a5f91a11d0
Fix older migrations on Ruby 3 (#16174) 2021-05-07 15:56:45 +02:00
Claire 566fc90913
Add Ruby 3.0 support (#16046)
* Fix issues with POSIX::Spawn, Terrapin and Ruby 3.0

Also improve the Terrapin monkey-patch for the stderr/stdout issue.

* Fix keyword argument handling throughout the codebase

* Monkey-patch Paperclip to fix keyword arguments handling in validators

* Change validation_extensions to please CodeClimate

* Bump microformats from 4.2.1 to 4.3.1

* Allow Ruby 3.0

* Add Ruby 3.0 test target to CircleCI

* Add test for admin dashboard warnings

* Fix admin dashboard warnings on Ruby 3.0
2021-05-06 14:22:54 +02:00
Eugen Rochko 036556d350
Fix media processing getting stuck on too much stdin/stderr (#16136)
* Fix media processing getting stuck on too much stdin/stderr

See thoughtbot/terrapin#5

* Remove dependency on paperclip-av-transcoder gem

* Remove dependency on streamio-ffmpeg gem

* Disable stdin on ffmpeg process
2021-05-05 19:44:01 +02:00
Shubhendra Singh Chauhan c8d11b8bdb
Fixed code quality issues (#15541)
* Added .deepsource.toml

* Removed bad use of `alias`

* Fixed operand order in the binary expression

* Prefixed unused method arguments with an underscore

* Replaced the old OpenSSL algorithmic constants with the newer strings initializers.

* Removed unnecessary UTF-8 encoding comment
2021-01-31 21:26:09 +01:00
luigi 087ed84367
Optimize map { ... }.compact calls (#15513)
* Optimize map { ... }.compact

using Enumerable#filter_map, supported since Ruby 2.7

* Add poyfill for Enumerable#filter_map
2021-01-10 00:32:01 +01:00
abcang efffdd3778
Fix rubocop config and warnings (#15503)
* disable NewCops

* update TargetRubyVersion

* Fix Lint/MissingSuper for ActiveModelSerializers::Model

* Fix Lint/MissingSuper for feed

* Fix Lint/FloatComparison

* Do not use instance variables
2021-01-07 09:40:55 +01:00
Eugen Rochko 1045549f85
Add stoplight for object storage failures, return HTTP 503 (#13043) 2020-12-15 12:55:29 +01:00
ThibG 5bbc9a4f78
Fix downloading remote media files when server returns empty filename (#14867)
Fixes #14817
2020-09-27 22:00:43 +02:00
Eugen Rochko e6b272e5c9
Change REST API to return empty data for suspended accounts (#14765) 2020-09-11 15:16:29 +02:00
dependabot[bot] 8972e5f7f6
Bump rubocop from 0.86.0 to 0.88.0 (#14412)
* Bump rubocop from 0.86.0 to 0.88.0

Bumps [rubocop](https://github.com/rubocop-hq/rubocop) from 0.86.0 to 0.88.0.
- [Release notes](https://github.com/rubocop-hq/rubocop/releases)
- [Changelog](https://github.com/rubocop-hq/rubocop/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rubocop-hq/rubocop/compare/v0.86.0...v0.88.0)

Signed-off-by: dependabot[bot] <support@github.com>

* Fix for latest RuboCop

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2020-09-01 03:04:00 +02:00
ThibG 0ebc8d8626
Fix thumbnail color extraction (#14464)
* Fix contrast calculation for thumbnail color extraction

Luminance calculation was using 0-255 RGB values instead of 0-1 sRGB values,
leading to incorrectly-computed contrast values.

Since we use ColorDiff already, just use its XYZ colorspace conversion code
to get the value.

* Require at least 3:1 contrast for both accent and foreground colors

* Lower required contrast for the accent color
2020-08-02 18:47:44 +02:00
ThibG a1412491b7
Change content-type to be always computed from file data (#14452)
* Change content-type to be always computed from file data

Restore previous behavior, detecting the content-type isn't very
expensive, and some instances may serve files as application/octet-stream
regardless of their true type, making fetching media from them fail, while
it used to work pre-3.2.0.

* Add test
2020-08-02 11:21:10 +02:00
Takeshi Umeda 101485a41f
Fix mimetype returning nil (#14356) 2020-07-19 13:09:47 +02:00
ThibG 701e5b9a19
Fix ogg vorbis files with a cover art not being correctly processed (#14255) 2020-07-07 16:32:14 +02:00
Eugen Rochko 99f3a55540
Add color extraction for audio thumbnails (#14209) 2020-07-05 18:28:25 +02:00
Eugen Rochko 534da9ba23
Fix audio uploads without embedded image (#14203) 2020-07-03 03:05:32 +02:00
Eugen Rochko 7aaf2b44ec
Fix remote files not using Content-Type header, streaming (#14184) 2020-06-30 23:58:02 +02:00
Eugen Rochko 64aac30733
Add customizable thumbnails for audio and video attachments (#14145)
- Change audio files to not be stripped of metadata
- Automatically extract cover art from audio if it exists
- Add `thumbnail` parameter to `POST /api/v1/media`, `POST /api/v2/media` and `PUT /api/v1/media/:id`
- Add `icon` to represent it in attachments in ActivityPub
- Fix `preview_url` containing URL of missing missing image when there is no thumbnail instead of null
- Fix duration of audio not being displayed on public pages until the file is loaded
2020-06-29 13:56:55 +02:00
ThibG 346d9b3d2e
Add blurhash to preview cards (#13984)
Fixes #13001
2020-06-05 23:10:41 +02:00
ThibG 927f9ea499
Fix GifReader exceptions (#13760) 2020-05-15 11:38:12 +02:00