1timspalding
Short version:
Between Saturday afternoon to Sunday morning spammers were able to insert "redirects" on work pages, sending users to terrible sites. We have removed the content and fixed the problem. We are monitoring for problems, working to find any other vulnerabilities, and repairing some lingering issues caused by our fix. We apologize for all of this.
See Spam Fix Bugs and Infelicities to discuss any bugs you see.
Long version:
Over the weekend, from around 4pm Saturday to 9am Sunday (EST), we had a spammer adding malicious content to work pages. Essentially they used the site's ability to add "Published Reviews" and "Quick Links" to redirect LibraryThing users to other sites. The other sites were the usual dreadful mix—porn, invitations to download apps, etc. This didn't involve any "back-end" compromise of LibraryThing systems, or to users' computers; they were just adding content to LibraryThing's web pages like any user. But the effect was extremely distressing to many members!
We are sorry to any users who were on the site at that time and were distressed. The problem is now fixed, and LibraryThing will behave normally. If you have any open tabs from the sites that LibraryThing sent you to, close them. I hope most of our users would never download an unknown application from the internet, still less because a flashing robot animation told them to, but if you did, you need to deal with that. The FTC's "How To Recognize, Remove, and Avoid Malware" is a good place to start.
Ultimately this is our fault. The fields in question should have more completely "cleaned" input, to screen out such problems. Malicious agents, especially scrapers and spammers, are a constant annoyance and threat to any website today, and we spend a lot of time and effort fighting them. We should have been aware of the potential issue here.
I'd like to thank the members of Spam Fighters who first noticed the problem, and helped me fix it.
What We've Done
While few users will notice, the initial things we did to "button up" the site have caused a few features to break. These include reordering Series entries. We also completely disabled all editing of Quick Links for now. We are working to fix these issues soon.
Questions?
If you have any questions, reply here, send me a message on LibraryThing, or email info@librarything.com.
I'd like to keep this thread mostly about the main issue—the spam attack. I have made a second Talk topic for "Spam Fix Bugs and Infelicities." We will be using that to circle up on the problems created by the fix, as it were.
Apologies again for this issue, and thank you for your support.
Between Saturday afternoon to Sunday morning spammers were able to insert "redirects" on work pages, sending users to terrible sites. We have removed the content and fixed the problem. We are monitoring for problems, working to find any other vulnerabilities, and repairing some lingering issues caused by our fix. We apologize for all of this.
See Spam Fix Bugs and Infelicities to discuss any bugs you see.
Long version:
Over the weekend, from around 4pm Saturday to 9am Sunday (EST), we had a spammer adding malicious content to work pages. Essentially they used the site's ability to add "Published Reviews" and "Quick Links" to redirect LibraryThing users to other sites. The other sites were the usual dreadful mix—porn, invitations to download apps, etc. This didn't involve any "back-end" compromise of LibraryThing systems, or to users' computers; they were just adding content to LibraryThing's web pages like any user. But the effect was extremely distressing to many members!
We are sorry to any users who were on the site at that time and were distressed. The problem is now fixed, and LibraryThing will behave normally. If you have any open tabs from the sites that LibraryThing sent you to, close them. I hope most of our users would never download an unknown application from the internet, still less because a flashing robot animation told them to, but if you did, you need to deal with that. The FTC's "How To Recognize, Remove, and Avoid Malware" is a good place to start.
Ultimately this is our fault. The fields in question should have more completely "cleaned" input, to screen out such problems. Malicious agents, especially scrapers and spammers, are a constant annoyance and threat to any website today, and we spend a lot of time and effort fighting them. We should have been aware of the potential issue here.
I'd like to thank the members of Spam Fighters who first noticed the problem, and helped me fix it.
What We've Done
- We fixed the direct problem, getting rid of all the bad content and preventing the addition of more.
- I have launched an effort to find any other vulnerable places where user content can be turned against the site.
- We are monitoring for suspicious activity.
- We have put some extra site-wide safeguards in place (see below).
While few users will notice, the initial things we did to "button up" the site have caused a few features to break. These include reordering Series entries. We also completely disabled all editing of Quick Links for now. We are working to fix these issues soon.
Questions?
If you have any questions, reply here, send me a message on LibraryThing, or email info@librarything.com.
I'd like to keep this thread mostly about the main issue—the spam attack. I have made a second Talk topic for "Spam Fix Bugs and Infelicities." We will be using that to circle up on the problems created by the fix, as it were.
Apologies again for this issue, and thank you for your support.
22wonderY
We applaud several features of this site that speeds up corrections.
- the trust you give to members to participate in policing many aspects of the site
- the ease in reaching staff to notify of any issues that need to be addressed. I was able to reach Abigail on a Saturday evening both by email and messaging LT’s Instagram account.
We are loyal members because you have built a site we love to hang out at; and we appreciate you and all your work.
Live long and prosper!
- the trust you give to members to participate in policing many aspects of the site
- the ease in reaching staff to notify of any issues that need to be addressed. I was able to reach Abigail on a Saturday evening both by email and messaging LT’s Instagram account.
We are loyal members because you have built a site we love to hang out at; and we appreciate you and all your work.
Live long and prosper!
3Charon07
>2 2wonderY: Hear, hear!
4SandraArdnas
>2 2wonderY: Wonderfully said. I co-sign it :)
5paradoxosalpha
Yay, Tim!
6laytonwoman3rd
Joining the chorus here...there's no other website that I use regularly that functions as well as this one.
8norabelle414
>1 timspalding: Thanks for being so responsive on a Saturday afternoon!
I second everything that >2 2wonderY: said, as well as adding my praise for such a well-designed helpers log and the incredible coincidence of Abigail adding published reviews to the helpers log a mere month and a half ago!
Not sure if this is the right thread for it but:
In addition to fixing the javascript-in-published-reviews issue, were you also able to fix whatever allowed the spammers to post tens of thousands of published reviews at once? That much activity alone really slowed down the Helpers Log and made it harder to find new spammers, even aside from the content of the reviews.
I second everything that >2 2wonderY: said, as well as adding my praise for such a well-designed helpers log and the incredible coincidence of Abigail adding published reviews to the helpers log a mere month and a half ago!
Not sure if this is the right thread for it but:
In addition to fixing the javascript-in-published-reviews issue, were you also able to fix whatever allowed the spammers to post tens of thousands of published reviews at once? That much activity alone really slowed down the Helpers Log and made it harder to find new spammers, even aside from the content of the reviews.
9timspalding
>8 norabelle414:
Yes. A good question. Not yet. Your solution came up in our all-tech meeting today. There are several solutions that achieve that result, and we are working through them.
Yes. A good question. Not yet. Your solution came up in our all-tech meeting today. There are several solutions that achieve that result, and we are working through them.
10PlumfieldLL
>1 timspalding: Thank you for all that you doing and are doing! This is such a fantastic resource because of all that you so!
11timspalding
If you're a member of the BETA (Board for Extreme Thing Advances) you'll see a note I wrote there:
https://www.librarything.com/topic/363461
If you're not a member, and are interested, go to https://www.librarything.com/ngroups/690/Board-for-Extreme-Thing-Advances and request to join.
https://www.librarything.com/topic/363461
If you're not a member, and are interested, go to https://www.librarything.com/ngroups/690/Board-for-Extreme-Thing-Advances and request to join.
12knerd.knitter
Quicklinks is back!
13timspalding
Quicklinks is back. You can't edit most of them—but can edit venues.
14SandraArdnas
>12 knerd.knitter: Open Library still throws a blank page saying just 'false'. Upon checking so do LoC, Project Gutenberg, and anything else I tried except Google Books and Amazon.
Some are missing links altogether, ISFDB for instance and others show 'ERROR: Name Missing' as a link. AFAICT from my sample, only Amazon and Google quick links work
Some are missing links altogether, ISFDB for instance and others show 'ERROR: Name Missing' as a link. AFAICT from my sample, only Amazon and Google quick links work
15knerd.knitter
>14 SandraArdnas:
I should have explained: the table was basically destroyed by the spammers, so we had to recreate it from a backup that was made about a year ago. Not everything will be there, but it was the best we could do quickly.
And the "ERROR: Name Missing" is ones that we couldn't repopulate.
I should have explained: the table was basically destroyed by the spammers, so we had to recreate it from a backup that was made about a year ago. Not everything will be there, but it was the best we could do quickly.
And the "ERROR: Name Missing" is ones that we couldn't repopulate.
16lilithcat
Don't know if this is another attempt, but see https://www.librarything.com/topic/356948#8626899
17civitas
>3 Charon07: Overzealous \ slashes showing up on Awards pages ... I'm hunting around for any new issues
I just found an HTML problem on my Profile page that may be related:
All the <img src="..."> tags that I coded have had a pair of \" characters (%5c%22) inserted at the beginning and end of the src attribute content string: <img src="%5c%22...%5c%22"> so the images would no longer load.
These links may have all begun with: https: which was dropped when the new characters were inserted around the remaining URL. Also, some img tags had height attributes that were stripped out when the tags were rewritten.
I was a bit impatient and fixed the HTML to see if the issue was caused by LT further limiting what was allowed in Profile page HTML. So, if you want to see an example in the wild, you'll need to find a Profile that uses the img tag and was similarly updated - I haven't found one. (Note: I restored one link to the way I found it, so you can see the effect - at the start of the About Me section.)
Hopefully, it's just the Profile page being updated as part of this fix. On the other hand, though, this issue might be a side effect of a different change.
I just found an HTML problem on my Profile page that may be related:
All the <img src="..."> tags that I coded have had a pair of \" characters (%5c%22) inserted at the beginning and end of the src attribute content string: <img src="%5c%22...%5c%22"> so the images would no longer load.
These links may have all begun with: https: which was dropped when the new characters were inserted around the remaining URL. Also, some img tags had height attributes that were stripped out when the tags were rewritten.
I was a bit impatient and fixed the HTML to see if the issue was caused by LT further limiting what was allowed in Profile page HTML. So, if you want to see an example in the wild, you'll need to find a Profile that uses the img tag and was similarly updated - I haven't found one. (Note: I restored one link to the way I found it, so you can see the effect - at the start of the About Me section.)
Hopefully, it's just the Profile page being updated as part of this fix. On the other hand, though, this issue might be a side effect of a different change.
18jasbro
>15 knerd.knitter: Will any of these eventually get repopulated? I'm particularly missing Bookfinder.com and WordCat.org.
19lesmel
>18 jasbro: bookfinder is there. Quick Links > Online > Booksellers. Do a page find (CTRL + f) for bookfinder. There are four on the list. Worldcat is there too. Same place.
20AnnieMod
>19 lesmel: All of them are there. But because they are now in Booksellers, they do not show up on the main work page - that's the part that got lost (the classification by type). And that is why none of them work anymore - as moving between the tabs changes them apparently.
21lesmel
>20 AnnieMod: What doesn't work? I've tried Worldcat and bookfinder QL. I'm finding things with the QLs. It's an extra click to Local Book Search before those QL show up.
22SandraArdnas
>21 lesmel: I can't even get WorldCat to show among quick links. Adding it does nothing once you go to work pages. It's till not even listed. IIRC the same was true for ISFDB, but moving it to databases fixed it, so perhaps that's all that's needed, at least for some.
23lesmel
>22 SandraArdnas: So, if you go here https://www.librarything.com/work/3429/get/183671848 and you have WorldCat added in the QLs, it doesn't show on the right?
24lesmel
>23 lesmel: Huh. Looks like someone moved Worldcat back to the Popular Databases tab and it's showing from the work page and not just the Local Book Search page.
25AnnieMod
>24 lesmel: See the last few messages here: https://www.librarything.com/topic/363374
26jasbro
>19 lesmel: As noted above, WorldCat on work pages is resolved. I've never really focused on Local Book Search, but when I go there now, I don't find either "Online > Booksellers," Bookfinder, or ISFDB, only "Bookstores," "Libraries," and "Swap Sites" (not including Bookfinder or ISFDB). And I can't tell how to move any of what I do see to "databases." What am I missing?
>25 AnnieMod: Seen! Thanks ...
>25 AnnieMod: Seen! Thanks ...
27knerd.knitter
Bookfinder and ISFDB are both added to databases now.
28mnleona
My first time to read these messages. LT tries so hard on their site and I find it one of the best sites I use. Thanks for all you do.
29jasbro
>27 knerd.knitter: Thank you! I don't (yet) see them on a work page. Is there some special magic to access them as databases?
30SandraArdnas
>29 jasbro: You have to readd them (and remove 'error name missing' ones, those are not coming back)
31jasbro
>30 SandraArdnas: Thanks! I just figured that out. I must have done it before at some, but haven't needed or used it since - and completely forgot about it.
