google spider?

[Jersey]

i always see google spider listed when i view the index.... WTH is a google spider?

 

[Tim]

Search engines use automated software programs that crawl the web. These programs called "crawlers" or "spiders" go from link to link and store the text and the keywords from the pages in a database. "Googlebot" is the name of Google's spider software.

 

[Jersey]

ok... so for instance this "google spider" was viewing thread "girls weekend"

does that mean if someone searches "gilrs weekend" our posts will show up??


yipes!!

 

[Tim]

While it is possible for the contents of that thread to show up via Google searching, it is highly unlikely.

Just remember that everything we post on the public side of this site can be found somewhere with search engines on the internet.

 

[Homer]

i thought it was something like that , we have two spiders but i don't remember the other ones full name.

 

[SilentEyz]

we get Yahoo spider and MSN spider alot too I have seen

 

[Jersey]

*begins deleting her posts*



hahaha--- ok.. it more of a curiosity thing... i hate not knowing things.. it woulda bugged me all day if i didnt ask

 

[lemon]

depending on the meta tags (of which, some are specifically meant for the "spiders"), a spider may index a page in the search engine's database.

however, i dont know how the undergound pages are built... since the spiders fall under the "guest" account type, i would think that they are not able to crawl in the underground.. but it all depends on the meta tags, as well as the programming of the logins and all that jazz...

 

[Bagel]

you can disable a spider's ability to crawl your pages with one line of code built into the meta so i imagine it wouldnt be horribly dififcult to block out a spider from a hosted pbb forum

 

[lemon]

true. i thought it was possible. maybe when crawlers first came out... but now..

edit - it is possible -

<meta name="robots" content="noindex,nofollow" />

Robots Exclusion Standard - Wikipedia, the free encyclopedia

and those that dont follow..

Clockwatchers Web Hosting - robots.txt Tutorial - Block Bad Bots

 

[Bagel]

true. i thought it was possible. maybe when crawlers first came out... but now..

edit - it is possible -
Robots Exclusion Standard - Wikipedia, the free encyclopedia

and those that dont follow..

Clockwatchers Web Hosting - robots.txt Tutorial - Block Bad Bots

told you... one line lol

i used to make websites..not professionally but i had to reteach myself html and i remember the section in the tutorial i was using about crawlers and meta-tags

 

[Jersey]

this thread means nothing to me now...



im lost

 

[Bagel]

ok.. lol.. heather..

search engines liek google,yahoo,hotspot,alta-vista etc..use software called crawlers or spiders that browse the interenet page by page and index them into a database that makes up the search engines log...webmasters can put a line of code into their pages to ensure that their sites don't get logged so people cannot find it thru a search engine..its good to keep certain sites private

 

[Dodge_Sniper]

This might help you out. When you make a site(Or in my case, a forum), usually you have to enter some words in a certain section that relate to your site. So for instance, on that section of my site, since it's SAW/Movie related, I'd type words like, "SAW, Jigsaw, Movie, DVD, Traps, etc.". And If I'm understanding this spider thing right, they read those words, and that way if you go to google and type in any of those words, the sight would be listed as a result. Try it, just go to google and type in my site name, SAWFreaks, you'll find that about 95% of the results are sites that somehow relate to me or OffTopicz(Since I've typed the word SAWFreaks on here many times before).

 

[SilentEyz]

this thread means nothing to me now...



im lost

Well If I am reading this all right.. and thats a big IF, then all your pics in the underground are safe from the spiders.

 

[lemon]

Well If I am reading this all right.. and thats a big IF, then all your pics in the underground are safe from the spiders.

if the spiders are guests. if they are members, and the site doesnt have a crawler "policy" .. then its a negative.

but since the crawlers are listed among guests, from my point of view, plus a little educational guessing magic, id say your pics are safe.

 

[Tim]

Well If I am reading this all right.. and thats a big IF, then all your pics in the underground are safe from the spiders.

You are correct, this site is set up so that even the spiders don't get to see the non member areas. Your posts in those areas are safe from search engines

 

[Veronica]

I personally like the google spiders looking at our sites... Its what helps generate traffic.

So its a good thing

 

[SilentEyz]

I personally like the google spiders looking at our sites... Its what helps generate traffic.

So its a good thing

Well hell yeah.. If it was not for Google I would have never found this place.. and then therfore as a result, my life would truly suck right now.

 

[Veronica]

im glad you found us.