Question

Bioconductor code browser time out

2

Entering edit mode

Marek Gierlinski ▴ 30

@mgierlinski-7369

Last seen 15 days ago

United Kingdom

Bioconductor code browser has been unresponsive for a while (at least since yesterday). When you try, for example,

https://code.bioconductor.org/browse/limma/RELEASE_3_18/

it munches through it for a while and returns "Bad gateway".

Bioconductor • 1.5k views

ADD COMMENT • link updated 3 months ago by Mike Smith ★ 6.6k • written 21 months ago by Marek Gierlinski ▴ 30

1

Entering edit mode

Thanks for the report. The site seems to being hit by a huge number of access attempts from a single IP and overloading it. I'm investigating the issue now.

ADD REPLY • link 21 months ago Mike Smith ★ 6.6k

0

Entering edit mode

It is back to normal now. Thank you.

ADD REPLY • link 21 months ago Marek Gierlinski ▴ 30

score 1 · Answer 1 · 2024-01-10

1

Entering edit mode

Mike Smith ★ 6.6k

@mike-smith

Last seen 19 days ago

EMBL Heidelberg

The site was being overwhelmed by various webcrawler bots, which were trying to index everything including all commits across all branches etc. There's potientially hundred of thousands of pages if you consider that, and I think the cumulative effect of those bots brought the site down.

I've now implemented a robots.txt which hopefully most of them will respect to try and limit what they access.

ADD COMMENT • link 21 months ago Mike Smith ★ 6.6k

0

Entering edit mode

Is this happening again? I've been getting "Gateway Timeout" messages more often than not when trying to use the code browser over the last month or so. Might just be a personal problem tho...

ADD REPLY • link 3 months ago dcwalker • 0

0

Entering edit mode

It's not just you. The site seems to be getting numerous requests from a wide variety of IP addresses. They don't seem to be webcrawlers (at least they don't identifify themselves as such) and it's more difficult for me to block them. I'm looking into solutions.

ADD REPLY • link 3 months ago Mike Smith ★ 6.6k