Robots Spiders and Bots

Forum for admin topics, member introductions and general non-hifi chitchat.
User avatar
Dr Bunsen Honeydew
Posts: 30758
Joined: Wed Jul 11, 2012 7:26 pm
Location: Muppet Labs
Has thanked: 0
Been thanked: 48 times

Robots Spiders and Bots

Unread post by Dr Bunsen Honeydew »

For some reason since the forum upgrade we have lost just about all of them. The only regular visitor now is Baidu from China. It has mystified Terry, does anyone here have knowledge of how we get them back.

Lurcher300b
Posts: 933
Joined: Tue Jun 14, 2016 1:58 pm
Has thanked: 0
Been thanked: 16 times

Re: Robots Spiders and Bots

Unread post by Lurcher300b »

Have you looked on the manage bot page in the phpbb admin section checking they are activated.

User avatar
terrybooth
Posts: 4396
Joined: Wed Jul 11, 2012 6:49 pm
Location: West Yorkshire
Has thanked: 499 times
Been thanked: 246 times
Great Britain

Re: Robots Spiders and Bots

Unread post by terrybooth »

Yes. All bots are activated.
Pioneer PL71/DL103/ Phono2/HiFiPi/P90SA/TIS/CubixPro

User avatar
terrybooth
Posts: 4396
Joined: Wed Jul 11, 2012 6:49 pm
Location: West Yorkshire
Has thanked: 499 times
Been thanked: 246 times
Great Britain

Re: Robots Spiders and Bots

Unread post by terrybooth »

94 guests online at the moment - looking at them, many appear to be bots. But they are not registered.
Pioneer PL71/DL103/ Phono2/HiFiPi/P90SA/TIS/CubixPro

User avatar
Dr Bunsen Honeydew
Posts: 30758
Joined: Wed Jul 11, 2012 7:26 pm
Location: Muppet Labs
Has thanked: 0
Been thanked: 48 times

Re: Robots Spiders and Bots

Unread post by Dr Bunsen Honeydew »

Ah! that would explain it, how do we register them or are they nasty fishing bots.

User avatar
terrybooth
Posts: 4396
Joined: Wed Jul 11, 2012 6:49 pm
Location: West Yorkshire
Has thanked: 499 times
Been thanked: 246 times
Great Britain

Re: Robots Spiders and Bots

Unread post by terrybooth »

The one's we want are the search engine ones. I've created a robots.txt file to see if that makes a difference.
Pioneer PL71/DL103/ Phono2/HiFiPi/P90SA/TIS/CubixPro

Lurcher300b
Posts: 933
Joined: Tue Jun 14, 2016 1:58 pm
Has thanked: 0
Been thanked: 16 times

Re: Robots Spiders and Bots

Unread post by Lurcher300b »

Does the bot page show the last time they visited?

User avatar
Dr Bunsen Honeydew
Posts: 30758
Joined: Wed Jul 11, 2012 7:26 pm
Location: Muppet Labs
Has thanked: 0
Been thanked: 48 times

Re: Robots Spiders and Bots

Unread post by Dr Bunsen Honeydew »

Yes. i just copied a few but only the top three have been here and only two today.

Baidu [Spider] Tue Nov 14, 2017 11:33 pm Deactivate Edit Delete
Exabot [Bot] Tue Nov 14, 2017 3:22 am Deactivate Edit Delete
Alexa [Bot] Fri Nov 10, 2017 6:49 am Deactivate Edit Delete
AdsBot [Google] Never Deactivate Edit Delete
Alta Vista [Bot] Never Deactivate Edit Delete
Ask Jeeves [Bot] Never Deactivate Edit Delete
Bing [Bot] Never Deactivate Edit Delete
FAST Enterprise [Crawler] Never Deactivate Edit Delete
FAST WebCrawler [Crawler] Never Deactivate Edit Delete
Francis [Bot] Never Deactivate Edit Delete
Gigabot [Bot] Never Deactivate Edit Delete
Google Adsense [Bot] Never Deactivate Edit Delete
Google Desktop Never Deactivate Edit Delete
Google Feedfetcher Never Deactivate Edit Delete

There is about 100 more listed as never visited. Obviously the formatting is wrong but I am sure you can see what I mean.

Lurcher300b
Posts: 933
Joined: Tue Jun 14, 2016 1:58 pm
Has thanked: 0
Been thanked: 16 times

Re: Robots Spiders and Bots

Unread post by Lurcher300b »

Try adding the website into googles search list.

https://www.google.com/webmasters/tools ... -url?pli=1

User avatar
Dr Bunsen Honeydew
Posts: 30758
Joined: Wed Jul 11, 2012 7:26 pm
Location: Muppet Labs
Has thanked: 0
Been thanked: 48 times

Re: Robots Spiders and Bots

Unread post by Dr Bunsen Honeydew »

Done, should everyone here do this at it seems to be an individual choice.

I have a thought, could it be they are all going to a Portal that is no longer there !!!!

Post Reply