Merge tag 'v4.1.1' into lets-bump-hometown-to-mastodon-4.2

Conflict resolution:

- ignored changed to README
- reverted all changes to the navigation panel, follow upstream
This commit is contained in:
nachtjasmin 2023-11-16 22:55:34 +01:00
commit f978ed560a
No known key found for this signature in database
31 changed files with 293 additions and 225 deletions

View File

@ -12,6 +12,7 @@ on:
- Dockerfile
permissions:
contents: read
packages: write
jobs:
build-image:
@ -26,15 +27,28 @@ jobs:
- uses: hadolint/hadolint-action@v3.1.0
- uses: docker/setup-qemu-action@v2
- uses: docker/setup-buildx-action@v2
- uses: docker/login-action@v2
- name: Log in to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
if: github.event_name != 'pull_request'
if: github.repository == 'mastodon/mastodon' && github.event_name != 'pull_request'
- name: Log in to the Github Container registry
uses: docker/login-action@v2
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
if: github.repository == 'mastodon/mastodon' && github.event_name != 'pull_request'
- uses: docker/metadata-action@v4
id: meta
with:
images: tootsuite/mastodon
images: |
tootsuite/mastodon
ghcr.io/mastodon/mastodon
flavor: |
latest=auto
tags: |
@ -42,13 +56,15 @@ jobs:
type=pep440,pattern={{raw}}
type=pep440,pattern=v{{major}}.{{minor}}
type=ref,event=pr
- uses: docker/build-push-action@v4
with:
context: .
platforms: linux/amd64,linux/arm64
provenance: false
builder: ${{ steps.buildx.outputs.name }}
push: ${{ github.event_name != 'pull_request' }}
push: ${{ github.repository == 'mastodon/mastodon' && github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max

View File

@ -3,6 +3,57 @@ Changelog
All notable changes to this project will be documented in this file.
## [4.1.1] - 2023-03-16
### Added
- Add redirection from paths with url-encoded `@` to their decoded form ([thijskh](https://github.com/mastodon/mastodon/pull/23593))
- Add `lang` attribute to native language names in language picker in Web UI ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/23749))
- Add headers to outgoing mails to avoid auto-replies ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/23597))
- Add support for refreshing many accounts at once with `tootctl accounts refresh` ([9p4](https://github.com/mastodon/mastodon/pull/23304))
- Add confirmation modal when clicking to edit a post with a non-empty compose form ([PauloVilarinho](https://github.com/mastodon/mastodon/pull/23936))
- Add support for the HAproxy PROXY protocol through the `PROXY_PROTO_V1` environment variable ([CSDUMMI](https://github.com/mastodon/mastodon/pull/24064))
- Add `SENDFILE_HEADER` environment variable ([Gargron](https://github.com/mastodon/mastodon/pull/24123))
- Add cache headers to static files served through Rails ([Gargron](https://github.com/mastodon/mastodon/pull/24120))
### Changed
- Increase contrast of upload progress bar background ([toolmantim](https://github.com/mastodon/mastodon/pull/23836))
- Change post auto-deletion throttling constants to better scale with server size ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/23320))
- Change order of bookmark and favourite sidebar entries in single-column UI for consistency ([TerryGarcia](https://github.com/mastodon/mastodon/pull/23701))
- Change `ActivityPub::DeliveryWorker` retries to be spread out more ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/21956))
### Fixed
- Fix “Remove all followers from the selected domains” also removing follows and notifications ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/23805))
- Fix streaming metrics format ([emilweth](https://github.com/mastodon/mastodon/pull/23519), [emilweth](https://github.com/mastodon/mastodon/pull/23520))
- Fix case-sensitive check for previously used hashtags in hashtag autocompletion ([deanveloper](https://github.com/mastodon/mastodon/pull/23526))
- Fix focus point of already-attached media not saving after edit ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/23566))
- Fix sidebar behavior in settings/admin UI on mobile ([wxt2005](https://github.com/mastodon/mastodon/pull/23764))
- Fix inefficiency when searching accounts per username in admin interface ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/23801))
- Fix duplicate “Publish” button on mobile ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/23804))
- Fix server error when failing to follow back followers from `/relationships` ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/23787))
- Fix server error when attempting to display the edit history of a trendable post in the admin interface ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/23574))
- Fix `tootctl accounts migrate` crashing because of a typo ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/23567))
- Fix original account being unfollowed on migration before the follow request to the new account could be sent ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/21957))
- Fix the “Back” button in column headers sometimes leaving Mastodon ([c960657](https://github.com/mastodon/mastodon/pull/23953))
- Fix pgBouncer resetting application name on every transaction ([Gargron](https://github.com/mastodon/mastodon/pull/23958))
- Fix unconfirmed accounts being counted as active users ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/23803))
- Fix `/api/v1/streaming` sub-paths not being redirected ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/23988))
- Fix drag'n'drop upload area text that spans multiple lines not being centered ([vintprox](https://github.com/mastodon/mastodon/pull/24029))
- Fix sidekiq jobs not triggering Elasticsearch index updates ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/24046))
- Fix tags being unnecessarily stripped from plain-text short site description ([c960657](https://github.com/mastodon/mastodon/pull/23975))
- Fix HTML entities not being un-escaped in extracted plain-text from remote posts ([c960657](https://github.com/mastodon/mastodon/pull/24019))
- Fix dashboard crash on ElasticSearch server error ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/23751))
- Fix incorrect post links in strikes when the account is remote ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/23611))
- Fix misleading error code when receiving invalid WebAuthn credentials ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/23568))
- Fix duplicate mails being sent when the SMTP server is too slow to close the connection ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/23750))
### Security
- Change user backups to use expiring URLs for download when possible ([Gargron](https://github.com/mastodon/mastodon/pull/24136))
- Add warning for object storage misconfiguration ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/24137))
## [4.1.0] - 2023-02-10
### Added

View File

@ -13,11 +13,7 @@ class BackupsController < ApplicationController
when :s3
redirect_to @backup.dump.expiring_url(10)
when :fog
if Paperclip::Attachment.default_options.dig(:fog_credentials, :openstack_temp_url_key).present?
redirect_to @backup.dump.expiring_url(Time.now.utc + 10)
else
redirect_to full_asset_url(@backup.dump.url)
end
redirect_to @backup.dump.expiring_url(Time.now.utc + 10)
when :filesystem
redirect_to full_asset_url(@backup.dump.url)
end

View File

@ -166,11 +166,19 @@ export function submitCompose(routerHistory) {
// API call.
let media_attributes;
if (statusId !== null) {
media_attributes = media.map(item => ({
id: item.get('id'),
description: item.get('description'),
focus: item.get('focus'),
}));
media_attributes = media.map(item => {
let focus;
if (item.getIn(['meta', 'focus'])) {
focus = `${item.getIn(['meta', 'focus', 'x']).toFixed(2)},${item.getIn(['meta', 'focus', 'y']).toFixed(2)}`;
}
return {
id: item.get('id'),
description: item.get('description'),
focus,
};
});
}
api(getState).request({

View File

@ -15,10 +15,10 @@ export default class ColumnBackButton extends React.PureComponent {
};
handleClick = () => {
if (window.history && window.history.length === 1) {
this.context.router.history.push('/');
} else {
if (window.history && window.history.state) {
this.context.router.history.goBack();
} else {
this.context.router.history.push('/');
}
};

View File

@ -43,14 +43,6 @@ class ColumnHeader extends React.PureComponent {
animating: false,
};
historyBack = () => {
if (window.history && window.history.length === 1) {
this.context.router.history.push('/');
} else {
this.context.router.history.goBack();
}
};
handleToggleClick = (e) => {
e.stopPropagation();
this.setState({ collapsed: !this.state.collapsed, animating: true });
@ -69,7 +61,11 @@ class ColumnHeader extends React.PureComponent {
};
handleBackClick = () => {
this.historyBack();
if (window.history && window.history.state) {
this.context.router.history.goBack();
} else {
this.context.router.history.push('/');
}
};
handleTransitionEnd = () => {

View File

@ -56,6 +56,8 @@ const messages = defineMessages({
redraftMessage: { id: 'confirmations.redraft.message', defaultMessage: 'Are you sure you want to delete this status and re-draft it? Favourites and boosts will be lost, and replies to the original post will be orphaned.' },
replyConfirm: { id: 'confirmations.reply.confirm', defaultMessage: 'Reply' },
replyMessage: { id: 'confirmations.reply.message', defaultMessage: 'Replying now will overwrite the message you are currently composing. Are you sure you want to proceed?' },
editConfirm: { id: 'confirmations.edit.confirm', defaultMessage: 'Edit' },
editMessage: { id: 'confirmations.edit.message', defaultMessage: 'Editing now will overwrite the message you are currently composing. Are you sure you want to proceed?' },
blockDomainConfirm: { id: 'confirmations.domain_block.confirm', defaultMessage: 'Hide entire domain' },
});
@ -149,7 +151,18 @@ const mapDispatchToProps = (dispatch, { intl, contextType }) => ({
},
onEdit (status, history) {
dispatch(editStatus(status.get('id'), history));
dispatch((_, getState) => {
let state = getState();
if (state.getIn(['compose', 'text']).trim().length !== 0) {
dispatch(openModal('CONFIRM', {
message: intl.formatMessage(messages.editMessage),
confirm: intl.formatMessage(messages.editConfirm),
onConfirm: () => dispatch(editStatus(status.get('id'), history)),
}));
} else {
dispatch(editStatus(status.get('id'), history));
}
});
},
onTranslate (status) {

View File

@ -21,8 +21,8 @@ const mapDispatchToProps = (dispatch) => ({
},
});
export default @connect(null, mapDispatchToProps)
@withRouter
export default @withRouter
@connect(null, mapDispatchToProps)
class Header extends React.PureComponent {
static contextTypes = {

View File

@ -1,10 +1,9 @@
import React from 'react';
import PropTypes from 'prop-types';
import { defineMessages, injectIntl } from 'react-intl';
import { connect } from 'react-redux';
import Avatar from 'mastodon/components/avatar';
import Permalink from 'mastodon/components/permalink';
import { timelinePreview, showTrends, me } from 'mastodon/initial_state';
import { Link } from 'react-router-dom';
import Logo from 'mastodon/components/logo';
import { timelinePreview, showTrends } from 'mastodon/initial_state';
import ColumnLink from './column_link';
import DisabledAccountBanner from './disabled_account_banner';
import FollowRequestsColumnLink from './follow_requests_column_link';
@ -12,7 +11,6 @@ import ListPanel from './list_panel';
import NotificationsCounterIcon from './notifications_counter_icon';
import SignInBanner from './sign_in_banner';
import NavigationPortal from 'mastodon/components/navigation_portal';
import { navRetracted } from 'mastodon/settings';
const messages = defineMessages({
home: { id: 'tabs_bar.home', defaultMessage: 'Home' },
@ -20,7 +18,6 @@ const messages = defineMessages({
explore: { id: 'explore.title', defaultMessage: 'Explore' },
local: { id: 'tabs_bar.local_timeline', defaultMessage: 'Local' },
federated: { id: 'tabs_bar.federated_timeline', defaultMessage: 'Federated' },
menu: { id: 'navigation_bar.menu', defaultMessage: 'Menu' },
direct: { id: 'navigation_bar.direct', defaultMessage: 'Direct messages' },
favourites: { id: 'navigation_bar.favourites', defaultMessage: 'Favourites' },
bookmarks: { id: 'navigation_bar.bookmarks', defaultMessage: 'Bookmarks' },
@ -29,26 +26,11 @@ const messages = defineMessages({
followsAndFollowers: { id: 'navigation_bar.follows_and_followers', defaultMessage: 'Follows and followers' },
about: { id: 'navigation_bar.about', defaultMessage: 'About' },
search: { id: 'navigation_bar.search', defaultMessage: 'Search' },
publish: { id: 'compose_form.publish', defaultMessage: 'Post' },
});
const Account = connect(state => ({
account: state.getIn(['accounts', me]),
}))(({ account }) => (
<Permalink className='column-link column-link--transparent navigation-panel--profile' href={account.get('url')} to={`/@${account.get('acct')}`} title={account.get('acct')}>
<Avatar account={account} size={32} inline />
<span>Profile</span>
</Permalink>
));
export default @injectIntl
class NavigationPanel extends React.Component {
constructor() {
super();
this.handleMenuToggle = this.handleMenuToggle.bind(this);
}
static contextTypes = {
router: PropTypes.object.isRequired,
identity: PropTypes.object.isRequired,
@ -58,93 +40,64 @@ class NavigationPanel extends React.Component {
intl: PropTypes.object.isRequired,
};
state = {
retracted: navRetracted.get('hometown'),
};
componentDidMount() {
const mainContent = document.querySelector('.columns-area--mobile');
if (this.state.retracted) {
mainContent.classList.add('fullWidth');
}
}
handleMenuToggle() {
this.setState({
retracted: !this.state.retracted,
}, () => navRetracted.set('hometown', this.state.retracted));
const mainContent = document.querySelector('.columns-area--mobile');
if (!this.state.retracted) {
mainContent.classList.add('navigation-panel--retracted');
mainContent.classList.remove('navigation-panel--extended');
} else {
mainContent.classList.add('navigation-panel--extended');
mainContent.classList.remove('navigation-panel--retracted');
}
};
render () {
const { intl } = this.props;
const { signedIn, disabledAccountId } = this.context.identity;
const isWideSingleColumnLayout = document.querySelector('.columns-area__panels__pane--compositional') && window.getComputedStyle(document.querySelector('.columns-area__panels__pane--compositional')).display !== 'none';
return (
<div className='navigation-panel'>
<ColumnLink transparent button onClick={this.handleMenuToggle} icon='bars' text={intl.formatMessage(messages.menu)} />
{ (isWideSingleColumnLayout || !this.state.retracted) && <div id='navigation-retractable'>
{signedIn && (
<React.Fragment>
<Account />
<ColumnLink id='navigation-panel__publish' transparent to='/publish' icon='pencil' text={intl.formatMessage(messages.publish)} />
<ColumnLink transparent to='/home' icon='home' text={intl.formatMessage(messages.home)} />
<ColumnLink transparent to='/notifications' icon={<NotificationsCounterIcon className='column-link__icon' />} text={intl.formatMessage(messages.notifications)} />
<FollowRequestsColumnLink />
</React.Fragment>
)}
{showTrends ? (
<ColumnLink transparent to='/explore' icon='search' text={intl.formatMessage(messages.explore)} />
) : (
<ColumnLink transparent to='/search' icon='search' text={intl.formatMessage(messages.search)} />
)}
{(signedIn || timelinePreview) && (
<>
<ColumnLink transparent to='/public/local' icon='users' text={intl.formatMessage(messages.local)} />
<ColumnLink transparent exact to='/public' icon='globe' text={intl.formatMessage(messages.federated)} />
</>
)}
{!signedIn && (
<div className='navigation-panel__sign-in-banner'>
<hr />
{ disabledAccountId ? <DisabledAccountBanner /> : <SignInBanner /> }
</div>
)}
{signedIn && (
<React.Fragment>
<ColumnLink transparent to='/conversations' icon='at' text={intl.formatMessage(messages.direct)} />
<ColumnLink transparent to='/favourites' icon='star' text={intl.formatMessage(messages.favourites)} />
<ColumnLink transparent to='/bookmarks' icon='bookmark' text={intl.formatMessage(messages.bookmarks)} />
<ColumnLink transparent to='/lists' icon='list-ul' text={intl.formatMessage(messages.lists)} />
<ListPanel />
<hr />
<ColumnLink transparent href='/settings/preferences' icon='cog' text={intl.formatMessage(messages.preferences)} />
</React.Fragment>
)}
<div className='navigation-panel__legal'>
<hr />
<ColumnLink transparent href='/about' icon='ellipsis-h' text={intl.formatMessage(messages.about)} />
</div>
<div className='navigation-panel__logo'>
<Link to='/' className='column-link column-link--logo'><Logo /></Link>
<hr />
</div>
{signedIn && (
<React.Fragment>
<ColumnLink transparent to='/home' icon='home' text={intl.formatMessage(messages.home)} />
<ColumnLink transparent to='/notifications' icon={<NotificationsCounterIcon className='column-link__icon' />} text={intl.formatMessage(messages.notifications)} />
<FollowRequestsColumnLink />
</React.Fragment>
)}
{showTrends ? (
<ColumnLink transparent to='/explore' icon='hashtag' text={intl.formatMessage(messages.explore)} />
) : (
<ColumnLink transparent to='/search' icon='search' text={intl.formatMessage(messages.search)} />
)}
{(signedIn || timelinePreview) && (
<>
<ColumnLink transparent to='/public/local' icon='users' text={intl.formatMessage(messages.local)} />
<ColumnLink transparent exact to='/public' icon='globe' text={intl.formatMessage(messages.federated)} />
</>
)}
{!signedIn && (
<div className='navigation-panel__sign-in-banner'>
<hr />
{ disabledAccountId ? <DisabledAccountBanner /> : <SignInBanner /> }
</div>
)}
{signedIn && (
<React.Fragment>
<ColumnLink transparent to='/conversations' icon='at' text={intl.formatMessage(messages.direct)} />
<ColumnLink transparent to='/bookmarks' icon='bookmark' text={intl.formatMessage(messages.bookmarks)} />
<ColumnLink transparent to='/favourites' icon='star' text={intl.formatMessage(messages.favourites)} />
<ColumnLink transparent to='/lists' icon='list-ul' text={intl.formatMessage(messages.lists)} />
<ListPanel />
<hr />
<ColumnLink transparent href='/settings/preferences' icon='cog' text={intl.formatMessage(messages.preferences)} />
</React.Fragment>
)}
<div className='navigation-panel__legal'>
<hr />
<ColumnLink transparent to='/about' icon='ellipsis-h' text={intl.formatMessage(messages.about)} />
</div>
}
<NavigationPortal />
</div>

View File

@ -475,10 +475,10 @@ class UI extends React.PureComponent {
};
handleHotkeyBack = () => {
if (window.history && window.history.length === 1) {
this.context.router.history.push('/');
} else {
if (window.history && window.history.state) {
this.context.router.history.goBack();
} else {
this.context.router.history.push('/');
}
};

View File

@ -162,6 +162,8 @@
"confirmations.discard_edit_media.message": "You have unsaved changes to the media description or preview, discard them anyway?",
"confirmations.domain_block.confirm": "Block entire domain",
"confirmations.domain_block.message": "Are you really, really sure you want to block the entire {domain}? In most cases a few targeted blocks or mutes are sufficient and preferable. You will not see content from that domain in any public timelines or your notifications. Your followers from that domain will be removed.",
"confirmations.edit.confirm": "Edit",
"confirmations.edit.message": "Editing now will overwrite the message you are currently composing. Are you sure you want to proceed?",
"confirmations.logout.confirm": "Log out",
"confirmations.logout.message": "Are you sure you want to log out?",
"confirmations.mute.confirm": "Mute",

View File

@ -259,6 +259,10 @@ html {
border-color: $ui-base-color;
}
.upload-progress__backdrop {
background: $ui-base-color;
}
// Change the background colors of statuses
.focusable:focus {
background: $ui-base-color;

View File

@ -4608,7 +4608,7 @@ a.status-card.compact:hover {
width: 100%;
height: 6px;
border-radius: 6px;
background: $ui-base-lighter-color;
background: darken($simple-background-color, 8%);
position: relative;
margin-top: 5px;
}

View File

@ -16,10 +16,7 @@ class PlainTextFormatter
if local?
text
else
node = Nokogiri::HTML.fragment(insert_newlines)
# Elements that are entirely removed with our Sanitize config
node.xpath('.//iframe|.//math|.//noembed|.//noframes|.//noscript|.//plaintext|.//script|.//style|.//svg|.//xmp').remove
node.text.chomp
html_entities.decode(strip_tags(insert_newlines)).chomp
end
end
@ -28,4 +25,8 @@ class PlainTextFormatter
def insert_newlines
text.gsub(NEWLINE_TAGS_RE) { |match| "#{match}\n" }
end
def html_entities
HTMLEntities.new
end
end

View File

@ -107,7 +107,7 @@ class Account < ApplicationRecord
scope :bots, -> { where(actor_type: %w(Application Service)) }
scope :groups, -> { where(actor_type: 'Group') }
scope :alphabetic, -> { order(domain: :asc, username: :asc) }
scope :matches_username, ->(value) { where(arel_table[:username].matches("#{value}%")) }
scope :matches_username, ->(value) { where('lower((username)::text) LIKE lower(?)', "#{value}%") }
scope :matches_display_name, ->(value) { where(arel_table[:display_name].matches("#{value}%")) }
scope :matches_domain, ->(value) { where(arel_table[:domain].matches("%#{value}%")) }
scope :without_unapproved, -> { left_outer_joins(:user).remote.or(left_outer_joins(:user).merge(User.approved.confirmed)) }

View File

@ -17,6 +17,6 @@
class Backup < ApplicationRecord
belongs_to :user, inverse_of: :backups
has_attached_file :dump, s3_permissions: ->(*) { ENV['S3_PERMISSION'] == '' ? nil : 'private' }
has_attached_file :dump, s3_permissions: 'private'
do_not_validate_attachment_file_type :dump
end

View File

@ -54,15 +54,15 @@
.strike-card__statuses-list__item
- if (status = status_map[status_id.to_i])
.one-liner
= link_to short_account_status_url(@report.target_account, status_id), class: 'emojify' do
= one_line_preview(status)
.emojify= one_line_preview(status)
- status.ordered_media_attachments.each do |media_attachment|
%abbr{ title: media_attachment.description }
= fa_icon 'link'
= media_attachment.file_file_name
- status.ordered_media_attachments.each do |media_attachment|
%abbr{ title: media_attachment.description }
= fa_icon 'link'
= media_attachment.file_file_name
.strike-card__statuses-list__item__meta
%time.formatted{ datetime: status.created_at.iso8601, title: l(status.created_at) }= l(status.created_at)
= link_to ActivityPub::TagManager.instance.url_for(status), target: '_blank' do
%time.formatted{ datetime: status.created_at.iso8601, title: l(status.created_at) }= l(status.created_at)
- unless status.application.nil?
·
= status.application.name

View File

@ -10,6 +10,16 @@ class ActivityPub::DeliveryWorker
sidekiq_options queue: 'push', retry: 16, dead: false
# Unfortunately, we cannot control Sidekiq's jitter, so add our own
sidekiq_retry_in do |count|
# This is Sidekiq's default delay
delay = (count**4) + 15
# Our custom jitter, that will be added to Sidekiq's built-in one.
# Sidekiq's built-in jitter is `rand(10) * (count + 1)`
jitter = rand(0.5 * (count**4))
delay + jitter
end
HEADERS = { 'Content-Type' => 'application/activity+json' }.freeze
def perform(json, source_account_id, inbox_url, options = {})

View File

@ -7,7 +7,7 @@ class Scheduler::AccountsStatusesCleanupScheduler
# This limit is mostly to be nice to the fediverse at large and not
# generate too much traffic.
# This also helps limiting the running time of the scheduler itself.
MAX_BUDGET = 50
MAX_BUDGET = 150
# This is an attempt to spread the load across instances, as various
# accounts are likely to have various followers.
@ -15,28 +15,22 @@ class Scheduler::AccountsStatusesCleanupScheduler
# This is an attempt to limit the workload generated by status removal
# jobs to something the particular instance can handle.
PER_THREAD_BUDGET = 5
PER_THREAD_BUDGET = 6
# Those avoid loading an instance that is already under load
MAX_DEFAULT_SIZE = 2
MAX_DEFAULT_SIZE = 200
MAX_DEFAULT_LATENCY = 5
MAX_PUSH_SIZE = 5
MAX_PUSH_SIZE = 500
MAX_PUSH_LATENCY = 10
# 'pull' queue has lower priority jobs, and it's unlikely that pushing
# deletes would cause much issues with this queue if it didn't cause issues
# with default and push. Yet, do not enqueue deletes if the instance is
# lagging behind too much.
MAX_PULL_SIZE = 500
MAX_PULL_LATENCY = 300
MAX_PULL_SIZE = 10_000
MAX_PULL_LATENCY = 5.minutes.to_i
# This is less of an issue in general, but deleting old statuses is likely
# to cause delivery errors, and thus increase the number of jobs to be retried.
# This doesn't directly translate to load, but connection errors and a high
# number of dead instances may lead to this spiraling out of control if
# unchecked.
MAX_RETRY_SIZE = 50_000
sidekiq_options retry: 0, lock: :until_executed
sidekiq_options retry: 0, lock: :until_executed, lock_ttl: 1.day.to_i
def perform
return if under_load?
@ -72,7 +66,6 @@ class Scheduler::AccountsStatusesCleanupScheduler
end
def under_load?
return true if Sidekiq::Stats.new.retry_size > MAX_RETRY_SIZE
queue_under_load?('default', MAX_DEFAULT_SIZE, MAX_DEFAULT_LATENCY) || queue_under_load?('push', MAX_PUSH_SIZE, MAX_PUSH_LATENCY) || queue_under_load?('pull', MAX_PULL_SIZE, MAX_PULL_LATENCY)
end

View File

@ -36,6 +36,7 @@ require_relative '../lib/terrapin/multi_pipe_extensions'
require_relative '../lib/mastodon/snowflake'
require_relative '../lib/mastodon/version'
require_relative '../lib/mastodon/rack_middleware'
require_relative '../lib/public_file_server_middleware'
require_relative '../lib/devise/two_factor_ldap_authenticatable'
require_relative '../lib/devise/two_factor_pam_authenticatable'
require_relative '../lib/chewy/strategy/mastodon'
@ -187,6 +188,10 @@ module Mastodon
config.active_job.queue_adapter = :sidekiq
config.action_mailer.deliver_later_queue_name = 'mailers'
# We use our own middleware for this
config.public_file_server.enabled = false
config.middleware.use PublicFileServerMiddleware if Rails.env.development? || ENV['RAILS_SERVE_STATIC_FILES'] == 'true'
config.middleware.use Rack::Attack
config.middleware.use Mastodon::RackMiddleware

View File

@ -16,12 +16,7 @@ Rails.application.configure do
# Run rails dev:cache to toggle caching.
if Rails.root.join('tmp/caching-dev.txt').exist?
config.action_controller.perform_caching = true
config.cache_store = :redis_cache_store, REDIS_CACHE_PARAMS
config.public_file_server.headers = {
'Cache-Control' => "public, max-age=#{2.days.to_i}",
}
else
config.action_controller.perform_caching = false

View File

@ -19,27 +19,16 @@ Rails.application.configure do
# or in config/master.key. This key is used to decrypt credentials (and other encrypted files).
# config.require_master_key = true
# Disable serving static files from the `/public` folder by default since
# Apache or NGINX already handles this.
config.public_file_server.enabled = ENV['RAILS_SERVE_STATIC_FILES'].present?
ActiveSupport::Logger.new(STDOUT).tap do |logger|
logger.formatter = config.log_formatter
config.logger = ActiveSupport::TaggedLogging.new(logger)
end
# Compress JavaScripts and CSS.
# config.assets.js_compressor = Uglifier.new(mangle: false)
# config.assets.css_compressor = :sass
# Do not fallback to assets pipeline if a precompiled asset is missed.
config.assets.compile = false
# `config.assets.precompile` and `config.assets.version` have moved to config/initializers/assets.rb
# Specifies the header that your server uses for sending files.
# config.action_dispatch.x_sendfile_header = 'X-Sendfile' # for Apache
config.action_dispatch.x_sendfile_header = 'X-Accel-Redirect' # for NGINX
config.action_dispatch.x_sendfile_header = ENV['SENDFILE_HEADER'] if ENV['SENDFILE_HEADER'].present?
# Allow to specify public IP of reverse proxy if it's needed
config.action_dispatch.trusted_proxies = ENV['TRUSTED_PROXY_IP'].split(/(?:\s*,\s*|\s+)/).map { |item| IPAddr.new(item) } if ENV['TRUSTED_PROXY_IP'].present?
@ -67,7 +56,7 @@ Rails.application.configure do
# Enable locale fallbacks for I18n (makes lookups for any locale fall back to
# English when a translation cannot be found).
config.i18n.fallbacks = [:en]
config.i18n.fallbacks = true
# Send deprecation notices to registered listeners.
config.active_support.deprecation = :notify
@ -128,6 +117,7 @@ Rails.application.configure do
enable_starttls_auto: enable_starttls_auto,
tls: ENV['SMTP_TLS'].presence && ENV['SMTP_TLS'] == 'true',
ssl: ENV['SMTP_SSL'].presence && ENV['SMTP_SSL'] == 'true',
read_timeout: 20,
}
config.action_mailer.delivery_method = ENV.fetch('SMTP_DELIVERY_METHOD', 'smtp').to_sym

View File

@ -12,11 +12,6 @@ Rails.application.configure do
# preloads Rails for running tests, you may have to set it to true.
config.eager_load = false
# Configure public file server for tests with Cache-Control for performance.
config.public_file_server.enabled = true
config.public_file_server.headers = {
'Cache-Control' => "public, max-age=#{1.hour.to_i}"
}
config.assets.digest = false
# Show full error reports and disable caching.

View File

@ -19,7 +19,6 @@ Chewy.settings = {
# cycle, which takes care of checking if Elasticsearch is enabled
# or not. However, mind that for the Rails console, the :urgent
# strategy is set automatically with no way to override it.
Chewy.root_strategy = :bypass_with_warning if Rails.env.production?
Chewy.request_strategy = :mastodon
Chewy.use_after_commit_callbacks = false

View File

@ -22,3 +22,5 @@ on_worker_boot do
end
plugin :tmp_restart
set_remote_address(proxy_protocol: :v1) if ENV['PROXY_PROTO_V1'] == 'true'

View File

@ -56,7 +56,7 @@ services:
web:
build: .
image: ghcr.io/mastodon/mastodon:v4.0.12
image: ghcr.io/mastodon/mastodon
restart: always
env_file: .env.production
command: bash -c "rm -f /mastodon/tmp/pids/server.pid; bundle exec rails s -p 3000"
@ -77,7 +77,7 @@ services:
streaming:
build: .
image: ghcr.io/mastodon/mastodon:v4.0.12
image: ghcr.io/mastodon/mastodon
restart: always
env_file: .env.production
command: node ./streaming
@ -95,7 +95,7 @@ services:
sidekiq:
build: .
image: ghcr.io/mastodon/mastodon:v4.0.12
image: ghcr.io/mastodon/mastodon
restart: always
env_file: .env.production
command: bundle exec sidekiq

View File

@ -372,16 +372,16 @@ module Mastodon
option :concurrency, type: :numeric, default: 5, aliases: [:c]
option :verbose, type: :boolean, aliases: [:v]
option :dry_run, type: :boolean
desc 'refresh [USERNAME]', 'Fetch remote user data and files'
desc 'refresh [USERNAMES]', 'Fetch remote user data and files'
long_desc <<-LONG_DESC
Fetch remote user data and files for one or multiple accounts.
With the --all option, all remote accounts will be processed.
Through the --domain option, this can be narrowed down to a
specific domain only. Otherwise, a single remote account must
be specified with USERNAME.
specific domain only. Otherwise, remote accounts must be
specified with space-separated USERNAMES.
LONG_DESC
def refresh(username = nil)
def refresh(*usernames)
dry_run = options[:dry_run] ? ' (DRY RUN)' : ''
if options[:domain] || options[:all]
@ -397,19 +397,25 @@ module Mastodon
end
say("Refreshed #{processed} accounts#{dry_run}", :green, true)
elsif username.present?
username, domain = username.split('@')
account = Account.find_remote(username, domain)
elsif !usernames.empty?
usernames.each do |user|
user, domain = user.split('@')
account = Account.find_remote(user, domain)
if account.nil?
say('No such account', :red)
exit(1)
end
if account.nil?
say('No such account', :red)
exit(1)
end
unless options[:dry_run]
account.reset_avatar!
account.reset_header!
account.save
next if options[:dry_run]
begin
account.reset_avatar!
account.reset_header!
account.save
rescue Mastodon::UnexpectedResponseError
say("Account failed: #{user}@#{domain}", :red)
end
end
say("OK#{dry_run}", :green)
@ -631,7 +637,7 @@ module Mastodon
exit(1)
end
unless options[:force] || migration.target_acount_id == account.moved_to_account_id
unless options[:force] || migration.target_account_id == account.moved_to_account_id
say('The specified account is not redirecting to its last migration target. Use --force if you want to replay the migration anyway', :red)
exit(1)
end

View File

@ -13,7 +13,7 @@ module Mastodon
end
def patch
0
1
end
def flags

View File

@ -0,0 +1,43 @@
# frozen_string_literal: true
require 'action_dispatch/middleware/static'
class PublicFileServerMiddleware
SERVICE_WORKER_TTL = 7.days.to_i
CACHE_TTL = 28.days.to_i
def initialize(app)
@app = app
@file_handler = ActionDispatch::FileHandler.new(Rails.application.paths['public'].first)
end
def call(env)
file = @file_handler.attempt(env)
# If the request is not a static file, move on!
return @app.call(env) if file.nil?
status, headers, response = file
# Set cache headers on static files. Some paths require different cache headers
headers['Cache-Control'] = begin
request_path = env['REQUEST_PATH']
if request_path.start_with?('/sw.js')
"public, max-age=#{SERVICE_WORKER_TTL}, must-revalidate"
elsif request_path.start_with?(paperclip_root_url)
"public, max-age=#{CACHE_TTL}, immutable"
else
"public, max-age=#{CACHE_TTL}, must-revalidate"
end
end
[status, headers, response]
end
private
def paperclip_root_url
ENV.fetch('PAPERCLIP_ROOT_URL', '/system')
end
end

View File

@ -23,7 +23,6 @@ describe Scheduler::AccountsStatusesCleanupScheduler do
},
]
end
let(:retry_size) { 0 }
before do
queue_stub = double
@ -33,7 +32,6 @@ describe Scheduler::AccountsStatusesCleanupScheduler do
allow(Sidekiq::ProcessSet).to receive(:new).and_return(process_set_stub)
sidekiq_stats_stub = double
allow(sidekiq_stats_stub).to receive(:retry_size).and_return(retry_size)
allow(Sidekiq::Stats).to receive(:new).and_return(sidekiq_stats_stub)
# Create a bunch of old statuses
@ -70,14 +68,6 @@ describe Scheduler::AccountsStatusesCleanupScheduler do
expect(subject.under_load?).to be true
end
end
context 'when there is a huge amount of jobs to retry' do
let(:retry_size) { 1_000_000 }
it 'returns true' do
expect(subject.under_load?).to be true
end
end
end
describe '#get_budget' do

View File

@ -942,15 +942,15 @@ const startWorker = async (workerId) => {
res.write('# TYPE connected_channels gauge\n');
res.write('# HELP connected_channels The number of Redis channels the streaming server is subscribed to\n');
res.write(`connected_channels ${Object.keys(subs).length}.0\n`);
res.write('# TYPE pg.pool.total_connections gauge \n');
res.write('# HELP pg.pool.total_connections The total number of clients existing within the pool\n');
res.write(`pg.pool.total_connections ${pgPool.totalCount}.0\n`);
res.write('# TYPE pg.pool.idle_connections gauge \n');
res.write('# HELP pg.pool.idle_connections The number of clients which are not checked out but are currently idle in the pool\n');
res.write(`pg.pool.idle_connections ${pgPool.idleCount}.0\n`);
res.write('# TYPE pg.pool.waiting_queries gauge \n');
res.write('# HELP pg.pool.waiting_queries The number of queued requests waiting on a client when all clients are checked out\n');
res.write(`pg.pool.waiting_queries ${pgPool.waitingCount}.0\n`);
res.write('# TYPE pg_pool_total_connections gauge\n');
res.write('# HELP pg_pool_total_connections The total number of clients existing within the pool\n');
res.write(`pg_pool_total_connections ${pgPool.totalCount}.0\n`);
res.write('# TYPE pg_pool_idle_connections gauge\n');
res.write('# HELP pg_pool_idle_connections The number of clients which are not checked out but are currently idle in the pool\n');
res.write(`pg_pool_idle_connections ${pgPool.idleCount}.0\n`);
res.write('# TYPE pg_pool_waiting_queries gauge\n');
res.write('# HELP pg_pool_waiting_queries The number of queued requests waiting on a client when all clients are checked out\n');
res.write(`pg_pool_waiting_queries ${pgPool.waitingCount}.0\n`);
res.write('# EOF\n');
res.end();
}));