Link: https://code.mschae23.de/mschae23/mlmym

hello. i'm a happy user of lemmy.today.

i've always used the mlmym frontend available on old.lemmy.today, it generally works well enough. however, there are some minor minor bugs and missing features. (e.g. impossible to navigate to next page when viewing saved posts, impossible to see down/upvotes separated)

i recently came across this fork of mlmym (from this github issue) and it seems to have all those bugs fixed. can we replace the current mlmym with this fork?

thanks for reading

Hello! I'm the person who made that fork.

If it does get set up here and you have any questions or run into any issues (which could very well happen, as I've done some major refactors), feel free to contact me, either by email at mlmym@lists.mschae23.de or in the #mlmym:mschae23.de matrix room.

You can also try it out already at discuss.mschae23.de (only logged-out here, of course, which doesn't give you all options) or by running it up locally with LEMMY_DOMAIN set to lemmy.today.

I'd also recommend enabling some of the options instance-wide by default, especially SHOW_UNREAD_COMMENTS (this one has been really useful for me).

Yes, let's do it.

And sorry for the late response, I didn't see this post until I got a private message about it.

Will try and get it up today.

Edit: And its up. I enabled the SHOW_UNREAD_COMMENTS option as well.

Thank you for your work on this. We are running it now on this instance. :)

thanks a lot! it seems to work well :)

edit: for some reason the communities bar on the top doesn't seem to work properly :( before there was a list of all subscripbed communities, now it's blank..

Hmm, that's not good... Could be a bug in this version perhaps... Will have to roll back if I can't fix it somehow.

i tried compiling it locally and there's the same issue. however it seems to appear in certain conditions but disappear again on a refresh..

now that we've rolled back, i'll let you know again if this bug gets solved.

thanks a lot

Ok see if you can report it to the author who posted in this thread. Once its fixed we can make another attempt. :)

Hm, it's working on my instance. I did change it to only make the search request for top communities once and cache it, so maybe that happened to run into an error and it's not trying again?

Running it locally on lemmy.today, it does seem to be working. I don't have an account here, unfortunately, so I can't test whether the “my communities” dropdown works too, but I don't see why it shouldn't.

There could have been another issue with traffic being blocked.

Will try again.

Perhaps it works now for you. I made some changes to the networking that was causing it to run slow, perhaps that also created your issues. If you hold shift and click reload in the browser to skip the browser cache, perhaps that will also work.

Is it the top bar in this picture you dont see?

Oh, while you're here, I think LANGUAGE_TAGS is also a good candidate to turn on by default (forgot to add it to the readme when I added the feature, so I also forgot about it in my earlier message...), and maybe SHOW_CONTROVERSIAL.

There's also EMBED_METADATA for embeds (for links on Discord and such) to show, but I don't know if you want that.

yes, the top bar (technology lemmyshitpost memes me_irl ...). it seems be working now for some reason, thanks

edit: hmm now comment scores aren't showing lmao. maybe there's something wrong with my browser?... seems like something's buggy...

hmm now comment scores aren't showing lmao. maybe there's something wrong with my browser?... seems like something's buggy...

That looks like you have “show scores” disabled in the preferences? Can't think of any other reason they wouldn't be showing like this.

Also, if you're wondering about the “also show on comments” option for upvote/downvote totals, I found that too distracting to show on every comment, so that extra information is only shown when there are actually both upvotes and downvotes.

I enabled all of them. :)

That looks like you have “show scores” disabled in the preferences? Can't think of any other reason they wouldn't be showing like this.

"show scores" was already enabled, but disabling -> saving -> enabling fixed it, i guess the settings for my account was in a weird state. it's all resolved now, thanks!

ok seems like i said that too early, the top bar disappearing issue came back, lol. it seems to work a few times randomly.. it isn't a critical bug so i'll keep using and investigating.

the 'my communities' button always seems to work

You said it only appears sometimes and it fixes itself on refreshes, right? I think that would rule out a mistake in the templates somewhere (which would've meant that it's consistently failing on some pages).

I wonder what the HTML output looks like when you can reproduce it. Do the <a> elements for the links not get output at all (more likely) or are they hidden somehow?

I'm not really sure on why this would happen, to be honest.

I wanted to ask you also if you could forward certain http headers to lemmy when it makes a request? Currently it doesnt forward them so lemmy thinks all the requests from mlmym to lemmy are from the docker ip its running on. And that makes it much harder to rate limit or ban bots since all requests are from the docker ip.

So basically, mlmym should preserve and forward X-Real-IP, X-Forwarded-For, X-Forwarded-Proto, and X-Forwarded-Host when proxying API/backend requests to Lemmy, because Lemmy uses these headers for correct client IP detection and rate limiting.

Something like this in the code where the incoming request is handled:

const forwardedFor = request.headers.get("x-forwarded-for");
const realIp =
  request.headers.get("x-real-ip") ??
  forwardedFor?.split(",")[0]?.trim();

const headers = new Headers(request.headers);

if (forwardedFor) {
  headers.set("x-forwarded-for", forwardedFor);
}

if (realIp) {
  headers.set("x-real-ip", realIp);
}

headers.set("x-forwarded-proto", request.headers.get("x-forwarded-proto") ?? "https");
headers.set("x-forwarded-host", request.headers.get("host") ?? "");

That would really help a lot and allow me and others to remove a lot of complicated workarounds for trying to get the source ip.

I have some handling for X-Forwarded-For, but none of the others. This could actually explain why I was still seeing rate limit errors despite not having made many requests from my IP address.

I'm unfortunately busy this week, but I hope I can still find some time in the next few days to implement this.

headers.set("x-forwarded-proto", request.headers.get("x-forwarded-proto") ?? "https");

Shouldn't that fallback be http, since mlmym doesn't do HTTPS itself? And I assume the reverse proxy will have set X-Forwarded-Proto to https.

Yeah, I have a custom setup currently just to try and get the source ip, but it doesn't work properly in all cases. And most of the bots use old.lemmy.today since they know mlmym doesn't forward the source ip address, and therefore it's hard to rate limit them.

So it would be amazing if you could forward those 4 headers. Then I will setup nginx to forward those headers to mlmym, and mlmym will forward them when making a request to Lemmy.

I actually thought it was making a https connection to Lemmy but perhaps it doesn't, and nginx just forwards http to https. Not sure. Will have to look into this more.

But yeah, whenever you have time to add the headers, i will investigate more. Not super urgent but will be interesting to play with that later. And it will protect all Lemmy instances that use your fork, so thats good.

Then I will setup nginx to forward those headers to mlmym

If you configure it to set X-Forwarded-For, you should have at least that information now already.

I actually thought it was making a https connection to Lemmy but perhaps it doesn't

That depends on the setup. If you haven't already, you should definitely set LEMMY_INTERNAL_URL so that mlmym connects to lemmy directly without TLS or a reverse proxy, which should be a lot faster (but that's only possible if they're running on the same machine, of course).

X-Forwarded-Proto is supposed to say what protocol the user (of mlmym) originally connected with though, right? Which will still end up being https in the common case of it being behind a reverse proxy that does TLS.

I will experiment more with this tomorrow or the coming days then. I was trying to get it to work previously but failed (not using your fork but the original mlmym).

Try to hold shift and reload the entire old.lemmy.today site in the browser. It's possible the old mlmym and this new one are conflicting somehow.

I also changed a lot of stuff with networking for it, so it's faster now I think.

If you still have problems, try a different browser just to see if it works there. Then we know it's a browser issue if it does.

I ended up using the external nginx for setting rules for rate limiting and such, and there I see the real ip. It's before traffic even gets to mlmym so that's good.

Also made sure to make it use the internal Lemmy url and skip a lot of the proxy stuff I was using earlier. It seems faster now.

I implemented the headers today, so version 26.2.1 should hopefully do this correctly now.

I could only really test it in my deployment, so feel free to let me know if you have any issues!

Cool. :) I think that will be a very useful feature for a lot of Lemmy admins.

Will play around with it later...!

It works! I discovered way too late that lemmy was rate limiting traffic from mlmym, because it wasnt part of my nginx config. It was lemmy itself that was doing rate limiting since all requests came from the same docker ip.

But forwarding the real ip headers fixed that, so very good. Huge thanks!

ok it's been a month, i now realized why scores don't show..

i use 'always private mode' in firefox, so my cookies get wiped on each and every session

cookies on this website isn't getting set when i just visit the main page (old.lemmy.today), i need to visit the settings page (old.lemmy.today/settings). just visiting settings once sets all the cookies and i'm good for that session..

i don't know programming properly but it seems like cookie setting code is only in here (settings), perhaps it should be added to other pages too?

still haven't figured out the top bar being empty in most cases tho..

i use 'always private mode' in firefox, so my cookies get wiped on each and every session

Oh, I see. I was able to reproduce the issue now.

cookies on this website isn't getting set when i just visit the main page (old.lemmy.today), i need to visit the settings page (old.lemmy.today/settings). just visiting settings once sets all the cookies and i'm good for that session..

i don't know programming properly but it seems like cookie setting code is only in here (settings), perhaps it should be added to other pages too?

The settings cookies are also supposed to be set when visiting the frontpage for the first time, but for some reason, only CookieAge is being set (maybe the GetSite call fails? figured it out. It was because CookieAge is also set when logged out, which is the reason the settings cookies weren't refreshed when loading the frontpage). But I just made it set these cookies on login, which I think is a good idea anyway.

The reason you only noticed this with scores is that the “show scores” setting is one of only a few that doesn't have an environment variable, so unlike most other settings, it thinks it's disabled rather than using the default when the cookie is missing. The same doesn't happen when logged out because the setting is just force-enabled in that case.

I just released version 26.3.1 that fixes this bug.

thanks a lot for the quick fix!

still haven't figured out the top bar being empty in most cases tho..

Do you happen to have set your listing type to local in the preferences? (“show posts from [local] communities by default”)

I was made aware of an issue with that today – basically, non-local communities are filtered out from the list when that is set, but the list only includes the top twenty communities in the first place, so it just ends up empty. Version 26.3.3 should fix this :)

indeed, i had that setting set. i tested the latest version locally and all seems well right now.

@mrmanager@lemmy.today could you please update the one on this instance?

Done! We are now on latest. :)