shazbot

joined 1 year ago
7
submitted 5 months ago* (last edited 5 months ago) by shazbot@kbin.social to c/kbinMeta@kbin.social
 

KES is the Kbin Enhancement Suite, a userscript/extension for (k/m)bin that provides a variety of customizable tools for users.

The last minor version included an initial attempt at a spam filter. This was chiefly done to address the low-hanging fruit of spam and scrub the most persistent pharma ads, etc. The approach was similar to that used by Ublock Origin or Steven Black's hosts file, in that it was a monolithic list of filter rules.

This was alright as a stopgap measure, but to the surprise of no one, the types of spam that continued to appear were innumerable in variety.

In an effort to have some filtering rather than none, the feature also went against the KES dictum about giving users choice to tweak their own settings: it was an all-or-nothing filter.

This update introduces v2 of spam filtering. The old logic has been retired, but may make a comeback at some point. A new sidebar page titled "Spam" has been added, and this will be a central place for anti-spam features.

The first of these efforts is a new filter that exposes the following options:

  • Hide posts from very new accounts

Users commented that there is no "gating" mechanism with registration, causing bogus accounts to be constantly recreated as a form of ban evasion. If you find that very new accounts have a high tendency to be spam, you can tick this setting to remove posts from very new users from the feed.

This does not block the user outright, merely hiding the post. This effectively gives some "break-in" time for the user to prove themselves. After a certain threshold, the user will fall outside of the "new account" window; if they continue to engage in spam-like behavior at that time, they should be caught by the other ban filters below.

  • Hide posts with abnormally low relative rank

Users with aggregate posting activity that does not necessarily resemble spam at first glance, but which periodically post spam/off-topic/controversial threads in the wrong communities, have a high likelihood of being astroturfers. This option hides such posts from the feed if the relative vote weight is egregiously outside the norm. Like the above, it does not block them outright.

  • Block users with spam-like activity

This feature blocks users outright if they engage in spam, mass posting, and other robotic behavior. It is less permissive than v1 of the spam filters, which failed to catch a lot of spam that was not explicitly blacklisted in the filter list. This new approach is more generic and does not need constant updating of the spam filters.


By default, all three options are enabled. I find this combination gives the best results so far. Note that infinite scrolling should be enabled for best results, and it may take a little while when navigating to a magazine for the results to be filtered.

Give it a try and see if this negates the spam issue or improves your experience. What other filter options would you like to see added?

The remainder of the changelog follows:

Add-ons


Filter advertisements (@shazbot)

Location: Spam > Filter advertisements

Updated filtering logic to v2, as described in the preamble of this post.

Collapse pinned posts (@shazbot)

Location: Threads > Collapse pinned posts

This simple feature groups together pinned posts on the magazine index and collapses them by default. Click the toggle area at the top of the magazine index to expand pinned posts. This is especially useful on magazines with many pinned posts that are not regularly archived, such as /m/kbinmeta.

API


Added the helper function isThread() to the API. This returns true if the current window location is a thread inside of a magazine. This can be used to abort if the feature is intended to only apply on the thread index. In the future, a more expanded function will be provided that returns the type of page from a list of enums.

[–] shazbot@kbin.social 3 points 6 months ago

If your main goal is exporting your magazine subscriptions between accounts across instances, may I suggest trying EXIT tool. If you are looking for more complex export settings (friends/favorites?), unfortunately, only subscriptions are supported at this time.

[–] shazbot@kbin.social 2 points 6 months ago

A few suggestions that may or may not be satisfactory for you:

Using KES, enable General > Hide sidebar elements > Random threads, Random mags, Random posts. The randomly populated sidebar is fundamentally flawed; I suggest disabling its content altogether.

Next, enable General > Filter advertisements. This second feature is by no means foolproof, but will reduce a lot of noise, and is periodically updated on a rolling basis.

[–] shazbot@kbin.social 5 points 6 months ago

Using this approach, I am seeing none of those posts on /science. I updated the filters a bit today. The top post is a legitimate article from 2024-04-13 and is by HeartyBeast.

Now, I understand that this is seen as an unnecessary step (too fancy) for some. People want zero ads out of the box without anything extra. So I'm thinking about the next approach here.

Framing the problem:

  • Filtering should be automatic
  • End-user wants zero additional setup
  • There is no active upstream development
  • It's not possible to inherit moderation of a magazine due to some queue of moderator application requests that is not being approved

The third point and fourth points are important here, since that's currently intractable. You can't reconcile zero additional setup with that.

But let's suppose becoming moderator of a defunct magazine (point 4) were possible while point 3 remained unresolved. In other words, at least moderators can try to pick up the pieces. Something being underestimated here is how annoying it would be for the moderator to manually cull posts every single day. I think you would have instant turnover after a couple of weeks once the tedium sets in. Manual solution is not good. Clearly, automation is needed on the moderation side.

So assuming you could actually inherit a magazine, but with no guarantee of upstream development, what about restructuring the tool above so that it's for moderators, instead of end-users? That's pretty easy, and I could make it something the moderator clicks once and it's done, auto-banning the posts. This is a pretty good method.

But you can't inherit moderation right now, so that's back to square one.

Realistically, that leaves these options at the moment:

  • Wait (a long time) and see
  • Use the tool above and make magazines readable, albeit at some sacrifice of convenience (?)
  • Migrate to another instance

Third approach is the path of least resistance and is best for most casual users. Second is for diehards who cannot move instances due to some personal or technical reason. First approach is the most annoying and eventually leads to the third approach after frustration sets in.

Pick your poison, I guess. I can't think of any other prophylactic approach at the moment, maybe this comment triggers some idea.

[–] shazbot@kbin.social 1 points 6 months ago* (last edited 6 months ago)

If the script can automatically block any user whose post it suppresses, it would be awesome.

It does! I've reworded the OP to hopefully make that clearer. After using this approach for a few days, my blocklist (generated entirely programmatically) is ten pages long, and there is nary a bad post in sight. I'm expanding the filters on a daily basis.

I think the auto report function is severely needed; it's happening everywhere.

The idea is that it takes the burden off of myriad (N) users having to manually do this themselves, and lets a single user (the KES custodian) prepare the filters, which then propagate out to any user of KES. Instead of 1,000 people manually blocking, one person builds the heuristics, and everyone benefits.

Preventing this issue doesn't seem like a userscript issue...but I think the issue is that we need to get support top-down on this.

I understand, but the stated goal of KES is addressing issues that can't, or won't (due to some design conflict), be addressed, or which fall through the cracks. At the moment I'm seeing a lot of people voicing frustration, but due to the skeleton crew situation with administration of the site, it seems like screaming into the void. Not that there's anything wrong with that, and hopefully it gets some traction. But my job with KES is just to provide fixes for the end-user, albeit of a third-party nature.

[–] shazbot@kbin.social 1 points 6 months ago (1 children)

I understand if you don't want to add yet another third-party tool just to browse the site, but if you feel at all inclined, give it a try. For me 99% of magazines went from unusable to essentially clean.

[–] shazbot@kbin.social 2 points 6 months ago

I don't read magazines in-depth much either, so I wasn't aware of the extent of the issue at first, but I was appalled at what I saw. I agree with you that it creates a negative impression for anyone wanting to venture into or use a magazine. I know that needing KES upfront may be a non-starter for some people, but for me the difference with filtering is night and day now.

[–] shazbot@kbin.social 1 points 6 months ago

@bayaz @insomniac_lemon

Yep, I meant to say /books when I was talking about Amazon links. Sorry.

As for the posts on /food, I have totally vanilla settings (no language filtering or anything) and I saw all of the posts you both mentioned.

Well, with the exception of the garbage posts, because in answer to lemon's earlier question, those have long since been scrubbed on the filter side. But they did exist before I started filtering.

All of that low hanging fruit kind of stuff has been banished, and I am mainly working on edge cases at this point.

[–] shazbot@kbin.social 2 points 6 months ago (15 children)

Of course, I'm not trying to suggest that a third-party prophylactic tool is a definitive solution to what is ultimately a separate problem, just trying to be pragmatic here and restore basic readability for end-users, whether the filtering is done at the source or after the fact.

Let's be real here, we are talking about unmoderated magazines on an instance where the developer is AWOL and using a framework that is lacking many basic features. Even with moderators, manual moderation can be a big ask and is time-consuming for free volunteers, depending on the volume of posts or how rudimentary the moderation tools are.

I actually don't read kbin magazines much, so I wasn't aware of the extent of the problem until I started opening those magazines more closely, and felt that something is better than nothing.

On the magazines you mentioned, I do see a few anomalous patterns that I'll start filtering. For the most part, with filtering enabled, they were almost entirely free of garbage, save for a few patterns I may have missed on the first few passes. /programming and /food I need to take a deeper look at. The /food thing is good intel, because the use of Amazon referral links in the threads is something that can be generalized to other situations beyond books. Posting referral links is definitively block-worthy.

I also noticed some stuff that by any other name would be considered a thinly-veiled ad, such as specific users only posting articles to web sites they own and operate. I'm not talking about bots as such, but actively promoting one's own content--even when such content is on-topic for the magazine. I declined to filter this stuff yet, because it received a lot of upvotes and seemed to be received favorably, maybe because the readers felt it was at least germane to the topic at hand? I think this is probably true for /food as well, because the line between "content" and "promotion" is unclear here, since what is a food blog if not a product generating click revenue? It seems like the tolerance threshold for that sort of thing is higher in a magazine like /food versus some other magazine. Anyway, I digress. I'm not treating such stuff as in scope, just filtering what is blatantly noise.

[–] shazbot@kbin.social 2 points 6 months ago (17 children)

You don't like bufo toad venom? I like to start my mornings by sipping a little bufo toad venom while reading kbin. Buy bufo toad venom today.

In all seriousness...I don't know if you saw my last thread about KES in this magazine, but I suggest giving it a try. I've extended the filter coverage based on your feedback, and those magazines should essentially be expunged of garbage for the time being.

As for the sidebar, I believe the implementation is fundamentally flawed because it loads content that, AFAIK, doesn't respect your actual block settings. I suggest disabling the random threads element altogether in KES by navigating to General > Hide sidebar elements.

[–] shazbot@kbin.social 2 points 6 months ago (20 children)

Could you point me to some of the magazines where you feel this is particularly rampant right now?

[–] shazbot@kbin.social 1 points 6 months ago (3 children)

How do you typically update your KES?

 

The blurb below is excerpted verbatim from the release notes. For the full release notes, see here.

Repository: KES

Many of you are aware of the "canned meat" problem on kbin.social, with some magazines being inundanted with garbage posts.

The latest version of KES ships with an experimental new feature you can enable that attemps to filter these posts and block the users who posted them based on certain heuristics.

This feature is experimental, but I see a lot of users voicing frustration at the problem, so now seems like a good time to start collecting feedback. You can start using this feature immediately and it should not have any adverse effects, but its coverage is still being expanded.

You can find it under General > Filter advertisements. For best results, it should be used in conjunction with infinite scrolling enabled in the kbin sidebar, so that new content is loaded in as posts are removed.

As you navigate through a magazine, KES will remove offending posts from the index and then permanently block the user of the post. This feature is also preventive, as variations of posts made under different usernames will continue to be flagged. The goal is to avoid the tedious process of "whack-a-mole" and cull these posts without manual intervention.

Initially, KES will be removing posts from the index, but as it builds your blocklist up for you, such posts will stop appearing in the thread index altogether, and you should see the overall signal to noise ratio improving. Outside of your blocklist, subsequent posts that meet certain criteria will continue to be culled regardless or when or where they appear.

I am currently using /m/science and /m/opensource as a control. If you navigate to those magazines and compare the results before and after enabling this feature, the difference should be clear. After enabling the feature and scrolling all the way back to 2023, there should be few if any unsolicited ads on the page.

Hopefully this improves readability and encourages participation in communities that otherwise seemed impenetrable at first glance. In fact, once you scrub the garbage posts, you'll be surprised to find that there are legitimate posts being made fairly frequently in these seemingly "dead" communities--the posts were just buried in the heap.

However, canned meat comes in a lot of different flavors, and each magazine has slightly different permutations. The coverage in this initial version is not exhaustive, but it attempts to be thorough. This should greatly cut down on the most annoying ads. If there are specific (most likely unmoderated) magazines you are still having a problem with, please leave a comment listing the magazine. You don't need to point to specific posts or users; the magazine name is enough here for me to analyze what kinds of posts are appearing.

Some additional notes:

  • For the time being, this feature does not report the post to the magazine's moderator (usually nonexistent). By kbin's design, a post can only be reported at most by a single user, so this seemed like a reduplication of efforts to me. But auto-report can be added if necessary.
  • This feature works on any instance, but is chiefly designed for kbin.social and is probably unnecessary elsewhere.
  • I have not taken a look at microblogs yet, so I don't know if this problem is happening there, too (please let me know). For now, this works on the thread index of magazines.
  • For best results (if you want to quickly bootstrap your blocklist), I suggest enabling the feature and scrolling through an affected magazine for awhile with infinite scroll on to build up the blocklist as new posts load in, then refreshing the page if necessary.
  • The "random threads" sidebar is fundamentally flawed because it shows content even if you've already blocked it. So I recommend enabling General > Hide sidebar elements > Random threads in conjunction with this feature.
[–] shazbot@kbin.social 1 points 7 months ago

Addressing your issue, I have bumped the version number to 0.1.3 and made a change to the async method handling so that instances not available at the remote get added to the fail log correctly.

This doesn't explicitly address the fact that some instances are unfederated, but it will make the log results clean.

As for the federation issue, what I've initially found is that a user on an instance has to visit the remote instance for the home instance to be aware of this remote instance, and a user (could be a different user) has to subscribe to that instance for the posts to start federating. What is unclear is how a user on an instance visits a remote instance from the home instance, as this is implementation-specific and could vary from instanc to instance.

 

KES is the Kbin Enhancement Suite.

This is just an update to note that the tool has undergone a total quality audit to ensure that all of its add-ons support both kbin and mbin instances and behave in an expected fashion regardless of which instance type you are using.

If you had previously tried KES on an mbin-type instance and encountered anomalous behavior, you should now find that it has full compatibility.

The full release notes can be found here

 

"EXIT" -- Export Across Instances Tool

This is a simple and self-contained tool that helps automate the process of exporting your magazine subscriptions from one instance to another, provided you have accounts on both.

Could also be used to copy subscriptions from one named account to another named account on the same instance, or to back them up for later.

Instructions and tool available here

Code runs locally in your browser only.

 

Original 1.0.0 release post here

Thanks to your detailed bug reports and the tireless efforts of our contributors, KES has undergone significant stability changes and has been upgraded with a more robust API framework to support the different flavors of GreaseMonkey, TamperMonkey, ViolentMonkey, etc.

If you had tried the previous version but could not get it working on GreaseMonkey/iOS, now is the time to try again.

In addition, this version brings a host of usability improvements, as well as a few new mods. Notably, this includes the Notifications Panel by @blobcat, which was highly asked for.

With this stability brush-up, this release paves the way for a stable foundation we can start adding many more mods to. If there is one you would like to see included, feel to drop a request here.


Notable feature additions:

  • Notifier on wrench icon if updates to KES are available
  • Transparent Mode: click the icon to see behind the KES menu and check changes on the page; click again to return
  • Reset button: clear all saved KES settings and reset
  • Clipboard button: copy system information to clipboard (used when submitting bug reports)
  • Notifications Panel (@blobcat): adds a navbar bell icon that opens notifications in an iframe
  • Bug-report-from-post: post contents of a message directly to the KES bug tracker
  • Display total number of add-ons enabled in header

Existing users:
The latest release is available through the install update button on KES, or through your extension manager.

New users:
The latest release is available here.
If you are a new user, see the docs for additional information and usage guides.


Mod authors:
This release ships with safeGM(), a shim which handles cross compatibility between the different GM APIs.
You can call this shim by passing it standard GM_ or GM. (4.0) commands without the prefix and pass the usual arguments.
A detailed explanation of this and other new utility functions is available here

There is also better support for fields like reset buttons, ranges (sliders), and number inputs. Now you can prompt users for a numerical input or tweak the value of a setting, or revert them to their initial values.

A total rewrite of the documentation now includes integration examples, sample code, and discussion of how KES handles
mutation observer events for you. This should make it even easier than before to port your scripts.

If you would like to contribute more actively to the development of KES, be it through testing, graphics, administrative issues, or code contributions, please feel free to reach out.

 

Recently I had been thinking about using localstorage to elaborate a settings menu that could be used to toggle various scripts that persist browser sessions and would make the process of script management less fiddly and prone to collisions for end-users.

Then @Perry made a subscriptions panel using precisely this sort of logic (a modal leveraging localstorage), so I had a turnkey logic ready to go, and gratefully made use of his modal implementation. Thank you!

kbin-megamod exposes a modal that covers the screen and could be used to fit a large amount of checkboxes (toggles) for different scripts, or any fancy menu-like styling. The idea is that by integrating scripts into a suite, users do not have to manage numerous disparate scripts, updates, and the potential for collisions between them, because they would be tested and integrated a priori into the suite and served from a unified endpoint.

An icon at the top right of the navbar to trigger the modal

The "Megamod Settings" panel exposes a series of toggles that trigger different scripts. This can cover the screen and could fit a large amount of scripts. Tabs or columns could be added as well.

The masthead of the script defines a series of includes (@require) that call a list of scripts sourced from the /mods directory of the same repository. For maintanability purposes, each "feature" integrated into the megamod should be an atomic script sourced here. The calling script itself only manages the menu and storage of settings. For security reasons, scripts should be peer reviewed and audited by contributors and served from a single location, reducing the likelihood of a remote script being changed to execute malicious code, or simply to introduce breaking changes. When the megamod is loaded, @require scripts are fetched and saved in local cache. (Note: when testing, if you wish to update the local cache, bump the version number of the megamod script to force a redownload.)

The business logic follows:

Three arrays and one object are declared and initialized. The first array contains a list of human-readable "labels" for each option in the menu:

/*human readable mod label*/
     const mmLabels = [
            "Add mail icon",
            "Add subs to navbar",
            "Label OP",
            "Profile dropdown",
        ];

The second array contains a human-readable list of descriptions for what each feature does.

    /*human readable mod desc*/
    const mmDescs = [
            "Add mail link to usernames if on kbin.social",
            "Add magazine subscriptions to navbar",
            "Add 'OP' label to thread author",
            "Convert profile page links to dropdown",
            ];

The third array contains the identifiers for the functions, and is used when populating DOM elements with a unique classname. These must be unique.

    /*function identifier, can be same as function name*/
    const mmFuncs = [
        "addMail",
        "initMags",
        "labelOp",
        "dropdownEntry"
        ];

Finally, an object maps these classnames to the actual function entrypoint in the cached script. This is merely used as a workaround for interpolating the classname variables returned from the event listener in order to call the cached function, as explained in the section after this. The entry point name need not be the same as the function identifier (classname) above, but must be unique to prevent collisions. For simplicity, I used entry points with the same name, but they could be anything, such as myEntryPoint.

The entry point logic in the target script must parse an incoming boolean and either enable or disable the logic accordingly. In the case of disabling, this means the script must tear down or hide any DOM elements previously created.

     /*object used for interpolation of function names*/
    /*key MUST be same as mmFuncs array*/
    /*value MUST be literal entry point in the target script, will be passed boolean*/
    /*literal func name need not be identical to key*/
      const funcObj = {
       addMail: addMail,
       initMags: initMags,
       labelOp: labelOp,
       dropdownEntry: dropdownEntry
       };

The toggle logic. The method argument is the name from the mmFuncs array that is returned when the event listener triggers on a checkbox. This is then used as a key in the funcObj and interpolated to the value (the actual script entry point) and a boolean argument is passed.

function applySettings(method) {
        const settings = getSettings();
        if (settings[method] == true) {
            funcObj[method](true);
        } else {
             funcObj[method](false);
        }
    }

The toggle logic in the destination script's entry point may look something like this:

function initMags(toggle){
    if (toggle === false) {
        $('.subs-nav').remove();
    } else {
        createMags();
    }
}

I believe this could be a way forward for curating a collection of scripts, or to use the language from the repository, a "megamod" collection of "mods." It should give end-users a more straightforward means of configuring their experience without having to collect and install disparate scripts, which may be difficult for some. (Of course, power users can continue to pick and choose from standalone scripts if they wish.)

Integrating a new feature is as simple as adding four lines to megamod.user.js

  • The feature label in the first array (mmLabels)
  • The feature description in the second array (mmDescs)
  • The classname in the third array (mmFuncs)
  • The entry point mapping in the final object (funcObj)

That means you only need to write two human-readable sentence fragments and two names, add the script to the /mods directory, and the script will be integrated in the modal. All of the business logic of the called feature is then handled by the cached script. Ensure that the script handles setup and teardown correctly when it is toggled and that the entry points are defined correctly. Also, don't forget to add jQuery to the @requires if necessary and any @grants or other remote content to the masthead of the megamod.user.js so that they are passed through.

Going forward, we should look into externalizing this into a manifest and preparing the arrays/objects programmatically, rather than inline.

I would like to ask for your contributions and suggestions. Below I am including callouts to users I saw had posted scripts here recently (from the first few pages). If you wish to integrate your script, please make a PR. Sorry if I missed anyone. The main area that will require some testing will be ensuring that the scripts "play nicely" with each other and that there is no reduplication of efforts or collisions. I believe most scripts are being MIT licensed, so they could be integrated anyway with proper attribution, but obviously having the script authors integrate their own code would be preferable.

The main thing you would have to refactor is add some teardown logic to your script so that when toggled to the off state, it reverts all of its changes and restores the elements to their prior state.

@raltsm4k, @minnieo, @blobcat, @artillect, @0rito, @SirPsychoMantis, @CodingAndCoffee

Postscript: as for the megamod script itself, I have not thoroughly cleaned everything yet. I have not fully integrated the "add dropdown" functionality from my other script, but merely added it as the last checkbox for testing purposes, so the cosmetic teardown is not working correctly when disabling it. Otherwise, the other toggles are working correctly and act as a minimum viable prototype. You can test enabling/disabling these features and should see the effects live.

Lastly, in case you are wondering, "won't these features be added to kbin upstream eventually?" If you are already a script author, I think you know the answer to this, but to clarify for other interested parties: there will always be a need for third-party modifications on any site, no matter how mature, as users may have wildly different needs. Changing things on the client-side is non-invasive and can be prototyped easily, as well as takes the strain on upstream developers off of UI concerns and lets them focus on more important issues. Finally, client-side scripts could also serve as a reference for designs used in production code, so these efforts are not wasted.

 

Works best in conjunction with:

 

Adds a direct 'Mail' link next to usernames if they are registered on the kbin instance

Image

Requires *monkey extension (greasemonkey, tampermonkey, violentmonkey et al)

 

Adds a direct 'Mail' link next to usernames if they are registered on the kbin instance

Image

 

This userscript replaces the superlong scrolling marquee of links on profile pages with a selection dropdown so that you can easily navigate to submitted posts, comments, followers, etc. Works on any profile page including your own.

Before
After

Edit: previous bug should be fixed on all versions now

 

This userscript replaces the superlong scrolling marquee of links on profile pages with a selection dropdown so that you can easily navigate to submitted posts, comments, followers, etc. Works on any profile page including your own.

Before
After

Requires monkey extension to run (greasemonkey, tampermonkey, violentmonkey, et al)

Edit: previous bugs should be fixed on all versions now

 

This userscript unsquashes inline images in comments by fetching the source image and downscaling it to 50%.

Requires *monkey extension to run (greasemonkey, tampermonkey, violentmonkey, et al)

Edit: also updated it to support the thread index

view more: next ›