* Add model for custom filter keywords * Use CustomFilterKeyword internally Does not change the API * Fix /filters/edit and /filters/new * Add migration tests * Remove whole_word column from custom_filters (covered by custom_filter_keywords) * Redesign /filters Instead of a list, present a card that displays more information and handles multiple keywords per filter. * Redesign /filters/new and /filters/edit to add and remove keywords This adds a new gem dependency: cocoon, as well as a npm dependency: cocoon-js-vanilla. Those are used to easily populate and remove form fields from the user interface when manipulating multiple keyword filters at once. * Add /api/v2/filters to edit filter with multiple keywords Entities: - `Filter`: `id`, `title`, `filter_action` (either `hide` or `warn`), `context` `keywords` - `FilterKeyword`: `id`, `keyword`, `whole_word` API endpoits: - `GET /api/v2/filters` to list filters (including keywords) - `POST /api/v2/filters` to create a new filter `keywords_attributes` can also be passed to create keywords in one request - `GET /api/v2/filters/:id` to read a particular filter - `PUT /api/v2/filters/:id` to update a new filter `keywords_attributes` can also be passed to edit, delete or add keywords in one request - `DELETE /api/v2/filters/:id` to delete a particular filter - `GET /api/v2/filters/:id/keywords` to list keywords for a filter - `POST /api/v2/filters/:filter_id/keywords/:id` to add a new keyword to a filter - `GET /api/v2/filter_keywords/:id` to read a particular keyword - `PUT /api/v2/filter_keywords/:id` to edit a particular keyword - `DELETE /api/v2/filter_keywords/:id` to delete a particular keyword * Change from `irreversible` boolean to `action` enum * Remove irrelevent `irreversible_must_be_within_context` check * Fix /filters/new and /filters/edit with update for filter_action * Fix Rubocop/Codeclimate complaining about task names * Refactor FeedManager#phrase_filtered? This moves regexp building and filter caching to the `CustomFilter` class. This does not change the functional behavior yet, but this changes how the cache is built, doing per-custom_filter regexps so that filters can be matched independently, while still offering caching. * Perform server-side filtering and output result in REST API * Fix numerous filters_changed events being sent when editing multiple keywords at once * Add some tests * Use the new API in the WebUI - use client-side logic for filters we have fetched rules for. This is so that filter changes can be retroactively applied without reloading the UI. - use server-side logic for filters we haven't fetched rules for yet (e.g. network error, or initial timeline loading) * Minor optimizations and refactoring * Perform server-side filtering on the streaming server * Change the wording of filter action labels * Fix issues pointed out by linter * Change design of “Show anyway” link in accordence to review comments * Drop “irreversible” filtering behavior * Move /api/v2/filter_keywords to /api/v1/filters/keywords * Rename `filter_results` attribute to `filtered` * Rename REST::LegacyFilterSerializer to REST::V1::FilterSerializer * Fix systemChannelId value in streaming server * Simplify code by removing client-side filtering code The simplifcation comes at a cost though: filters aren't retroactively applied anymore.
106 lines
3.1 KiB
Ruby
106 lines
3.1 KiB
Ruby
# frozen_string_literal: true
|
|
# == Schema Information
|
|
#
|
|
# Table name: custom_filters
|
|
#
|
|
# id :bigint not null, primary key
|
|
# account_id :bigint
|
|
# expires_at :datetime
|
|
# phrase :text default(""), not null
|
|
# context :string default([]), not null, is an Array
|
|
# created_at :datetime not null
|
|
# updated_at :datetime not null
|
|
# action :integer default(0), not null
|
|
#
|
|
|
|
class CustomFilter < ApplicationRecord
|
|
self.ignored_columns = %w(whole_word irreversible)
|
|
|
|
alias_attribute :title, :phrase
|
|
alias_attribute :filter_action, :action
|
|
|
|
VALID_CONTEXTS = %w(
|
|
home
|
|
notifications
|
|
public
|
|
thread
|
|
account
|
|
).freeze
|
|
|
|
include Expireable
|
|
include Redisable
|
|
|
|
enum action: [:warn, :hide], _suffix: :action
|
|
|
|
belongs_to :account
|
|
has_many :keywords, class_name: 'CustomFilterKeyword', foreign_key: :custom_filter_id, inverse_of: :custom_filter, dependent: :destroy
|
|
accepts_nested_attributes_for :keywords, reject_if: :all_blank, allow_destroy: true
|
|
|
|
validates :title, :context, presence: true
|
|
validate :context_must_be_valid
|
|
|
|
before_validation :clean_up_contexts
|
|
|
|
before_save :prepare_cache_invalidation!
|
|
before_destroy :prepare_cache_invalidation!
|
|
after_commit :invalidate_cache!
|
|
|
|
def expires_in
|
|
return @expires_in if defined?(@expires_in)
|
|
return nil if expires_at.nil?
|
|
|
|
[30.minutes, 1.hour, 6.hours, 12.hours, 1.day, 1.week].find { |expires_in| expires_in.from_now >= expires_at }
|
|
end
|
|
|
|
def irreversible=(value)
|
|
self.action = value ? :hide : :warn
|
|
end
|
|
|
|
def irreversible?
|
|
hide_action?
|
|
end
|
|
|
|
def self.cached_filters_for(account_id)
|
|
active_filters = Rails.cache.fetch("filters:v3:#{account_id}") do
|
|
scope = CustomFilterKeyword.includes(:custom_filter).where(custom_filter: { account_id: account_id }).where(Arel.sql('expires_at IS NULL OR expires_at > NOW()'))
|
|
scope.to_a.group_by(&:custom_filter).map do |filter, keywords|
|
|
keywords.map! do |keyword|
|
|
if keyword.whole_word
|
|
sb = /\A[[:word:]]/.match?(keyword.keyword) ? '\b' : ''
|
|
eb = /[[:word:]]\z/.match?(keyword.keyword) ? '\b' : ''
|
|
|
|
/(?mix:#{sb}#{Regexp.escape(keyword.keyword)}#{eb})/
|
|
else
|
|
/#{Regexp.escape(keyword.keyword)}/i
|
|
end
|
|
end
|
|
[filter, { keywords: Regexp.union(keywords) }]
|
|
end
|
|
end.to_a
|
|
|
|
active_filters.select { |custom_filter, _| !custom_filter.expired? }
|
|
end
|
|
|
|
def prepare_cache_invalidation!
|
|
@should_invalidate_cache = true
|
|
end
|
|
|
|
def invalidate_cache!
|
|
return unless @should_invalidate_cache
|
|
@should_invalidate_cache = false
|
|
|
|
Rails.cache.delete("filters:v3:#{account_id}")
|
|
redis.publish("timeline:#{account_id}", Oj.dump(event: :filters_changed))
|
|
redis.publish("timeline:system:#{account_id}", Oj.dump(event: :filters_changed))
|
|
end
|
|
|
|
private
|
|
|
|
def clean_up_contexts
|
|
self.context = Array(context).map(&:strip).filter_map(&:presence)
|
|
end
|
|
|
|
def context_must_be_valid
|
|
errors.add(:context, I18n.t('filters.errors.invalid_context')) if context.empty? || context.any? { |c| !VALID_CONTEXTS.include?(c) }
|
|
end
|
|
end
|