Member-uploaded photos not showing on work page

Also not in Member galleries, which I noticed when I took a screen shot of a work page to show you.

See https://www.librarything.com/pic/11270436 and https://www.librarything.com/work/81818/covers/275410471

Not showing up here either.

Pics seemed to be broken all over: author photos, book covers, member galleries, etc.

It's getting worse, too--more images are disappearing as the night goes on

Member Uploaded covers are not showing - only a broken photo icon. This worked yesterday and into early morning (CST). When attempting to change a cover, the previous member uploaded covers are broken. When uploading a new jpeg, it appears to upload and then shows a broken jpeg icon. Covers uploaded yesterday are broken on the work page. However, the covers uploaded before today do show up correctly on the page listing "Your books" while a cover photo uploaded today is also broken on the "Your books" page.

Additional Information: the very last cover photo that I uploaded at about 2-3 AM CST 0m 11/22/2024 shows a broken jpeg icon on BOTH the work page and the "Your books" page. All previous uploaded photos from early this morning are still OK on the "Your books" page but not on the work page. Maybe this will help sort out when the problem began.

My images that I uploaded have disappeared on my thread. I guess that is the case everywhere on LT right now.

Even on the LibraryThing App (Android)... There are lot of covers that are missing and some of those that are showing can't be viewed.
At least I know it isn't just me.

Confirming (desktop / Win10 / Firefox)

I'm experiencing this as well (MacOS/Chrome).

same here :0(

Pretty sure it's not only member-uploaded images, but also publisher provided ones (a variety of sources), and possibly also images from Amazon. I have begun tagging my books with broken links (LT imagehack), so far about 1/4 to 1/3 per page and I've gone through about six of my 17 pages. Hoping the problem gets solved before I finish tagging :(

I'm having the same issue, though it's only on book covers that were member-uploaded.

517 cover images disappeared out of 1631 entries, sigh. all tagged with "LT imagehack", in case that helps anyone working on a solution.

Thanks for the note. Looks like our image server went down overnight or CloudFlare is having a problem connecting to it. We are looking at it.

You really don't need to tag anything "LT imagehack." It's not a hack. It's not about anything other than LT images. And telling us which isn't going to help—we can find plenty of examples.

It's the weekend. Of course that's when things go wonky.

Covers not loading. Covers of old entries disappearing. Big fun.

LT has (infrequently) experienced server afflictions like this in the past. I have a lot of confidence that staff will sort out the problem by Monday at the latest. There's not much for us to do as users now except to endure a brief period of site ugliness.

I brought "Brødrene Karamasov Bind I by Fjodor M. Dostojevskij forward. The picture is failing so I tried to reinstall but it failed too.

https://www.librarything.com/work/1934851&coverredo=1434520723

Cann't the system take JPEG-files anymore?

>19 kirstenlund: Right now all sorts of images are failing. See >15 timspalding:.

I think the problem with image display is reported! We just have to wait confidently for the solution to the problem!

As I watched more and more user uploaded images fail to properly show up on various pages over the past day, it seems to me that the photo files are becoming corrupted. From my security engineering past, it reminds me of the behavior of ransomware software that systematically visits each directory and encrypts the files. If so, possible next steps: restore the files from backup (please), pay the ransom to get the encryption key (no), or have users upload files again (please no)! Hopefully this is just some sort of a database retrieval issue - software or hardware. Just some thoughts...

>22 Brewmaster: it seems to me that the photo files are becoming corrupted.

I see no sign of this. Either images load or they don't -- and if they do load, they are correct images. We've been told there are server problems. That doesn't mean corrupted files; it just means they aren't being found.

You may well be right that uploads are not being saved. But that's not corruption; that's absence.

>23 waltzmn: waltzmn, I hope you are correct!

Over the past day or so, by coincidence, I added and uploaded images for few dozen books. All was fine as of the wee hours of Friday morning.

When I started up again yesterday mid day, I was unable to successfully upload a file. I tried uploading jpegs that were successfully uploaded the previous but they failed now. Then I noticed the very last image from my previous session was now not showing - but all of the others were. Then more would not display correctly. Now, none of them display. However, images from Amazon do show correctly. Presumably, they are sourced from a different database server.

It was the progressive nature of the failure that led me to believe files were gradually becoming corrupt. If it was just a database access issue, then I believe that either all of the member uploaded covers would load or they would not. Now, it appears no member uploaded covers display which seems to me that encryption is complete.

Anyway, thanks for responding. It would be nice to know what the actual problem was when it is fixed. I really want your explanation of it being a database access issue to be true!

I'm so relieved to hear that (1) this is something that's happened before and (2) that experienced folks are looking into it. Thanks very very much! (PS I'm tagging for my own interest, too.)

I would recommend:
A) Not uploading cover images until the issue is resolved, and
B) Not worrying about data persistence/integrity for previous uploads.

I'm willing to wager all those images are still happy in LT, they just aren't being delivered to our browsers right now.

And now the newly uploaded cover uploads are slowly coming back into view!

I'm seeing all my previously disappeared covers back! Fingers-crossed! Thank you!

This seems to have been resolved. Hurrah! Will wait for staff to close the bug.

Thank you, thank you!

>28 airgid:, >29 lilithcat: Yep, same here! Thank you!

All my cover pictures are back. Thanks

All of my photos are back. Thanks , LT staff !

As members noticed, the images are all back.

I would really caution members against jumping to conclusions about hacks and corruption.

LibraryThing has dozens of different systems, internal to us an external to us, and they interact in various ways and can therefore fail in various ways.

In this case we have two image servers which makes the images and serves them up sometimes, a "load balancer" in front of that, which allocates work to them, and a commercial CDN (Content Delivery Network), which is an external galaxy of servers that cache static files, like images, and being located around the world, can deliver them up faster.

For most LT images, you don't get the image from LT, but from our CDN. When the CDN doesn't have the image, or when it's older than a certain amount of time, it asks the image servers—or rather asks the load balancers—first, then serves that.

In this case there was some problem with the "SSL Cert" on our servers. So if it had it cached, it delivered it, but if it didn't, or the caceh was old, it would try to get it from our image servers--via the load balancer--and would fail. When the problem happened, I was able to diagnose it as a SSL issue, but that's black magic as far as I'm concerned. Ganawa—our sysadmin—came on later and fixed it. He left an explanation, but it's black-magic enough that I can't explain it past that.

There seems to be a rash of stutters when members add profile pictures. Some are added multiple times and members don’t realize and delete the extras. I’ve noticed this for the past couple of weeks.
See the Recent members page:
https://www.librarything.com/gallery/recent

Not that this has anything to do with the server issue you’ve identified. But I wonder if it’s something you’d like to address. I know that images cost a lot of bytes and I appreciate that you allow them on the site.

>34 timspalding: it's an encore today....

Confirming

Our pics system is having problems right now. We are on the case and making an effort to throw more hardware at it.

>38 conceptDawg:

Just don't throw too hard and break the hardware!

>39 lilithcat: Seconded! I have learned from experience, hardware must be thrown very carefully, and very skillfully. Ouch...

Even when I take the photo with the Library Thing app it's not loading, Worked yesterday...

>41 teresameckley: See >38 conceptDawg: - LT are working on the images issues.

Update on this. I made updates to the image service code over the last week. We are phasing it in slowly (currently it's building 20% of requested images and it increases daily) because recreating all pics at one time would likely bring our pics system to its knees.

We also have a whole new pics server system (the non-code, server resources) ready to roll out in the next day or so. That should increase stability and improve our ability to scale the pics system as it grows.

Additionally, new features and bug fixes have been rolled out. I'll speak to those once we release the server-side improvements and we can talk about all of it in a single post.

Uploading Book covers are not showing again

Thanks now all the book-covers show.

This is happening again. Not all covers are showing.

ETA: now they're back.

ETA: now they're gone.

Something is causing this to happen sporadically.

Based on the info in >43 conceptDawg:, I suspect the instability we are seeing is a byproduct of the transition to the new server system, hopefully all to be resolved once that system is fully live.

Thanks. Yes. We've been having troubles with the pics system. Both ours and our CDN. ccatalfo and I are working hard to figure out why and how we can rebound quickly.

Friday the 13th and images taken in app or from my gallery on android mobile will not upload. It refuses them instantly. When will the issue be resolved please LT?

>49 Laurence_Batchelor: Just wondering - have you tried to click on "Recalculate cover" ?

>49 Laurence_Batchelor: Sorry for the trouble: can you try again, now, and let me know if you're still seeing any issues (and if so, where: from the app or on mobile web, etc.)? Our image issues have been pretty good since the weekend, although we are walking a bit of a tightrope at the moment. (We're also in the midst of getting some added support for images in the next few days, hopefully, which should make things even better.)

ETA: also, if you're having trouble with the Android App specifically it might be tied to this issue: https://www.librarything.com/topic/365409