Announcement

#1 2016-12-15 15:54:57

WuppiGER
Member
Germany
2016-05-31
38

Meta: robots -> noindex,nofollow @ pictures ...

Hello community ;)

i've tested a new theme (bootstrap) - but in failed for me. I've deactivate/delete this. After (!) this i see in my sourcecode: (for example: https://galerie.fotobastler.de/picture/219/categories )

<meta name="robots" content="noindex,nofollow">
(same on bootstrap-Theme or "dark" ... => not a theme-specific problem)

On every picture-Page

It's definitly new! See: site:galerie.fotobastler.de @google -> there a many pictures ...

How can i set it to index/follow?

This Problem isn't really new ... some months again i have the same - sometimes - not on every page-load! See: http://piwigo.org/forum/viewtopic.php?id=25710 ... But reload, reload-again doesn't work anymore ... grrrr ....

Piwigo version: 2.8.3
PHP version: 7
MySQL version: 5.5

Thx
Wuppi


Edit:
header.tpl ist clean ... no "no...."
The Noindex is generated by this line:

Code:

{if not empty($head_elements)}
{foreach from=$head_elements item=elt}
  {$elt}
{/foreach}
{/if}

Edit 2: Workaround [at the moment deactivated - but it works]

Code:

{if not empty($head_elements)}
{foreach from=$head_elements item=elt}
{if $elt|strstr:"noindex"} {continue}{/if} {* workaround for noindex/nofollow *}
{$elt} 
{/foreach}
{/if}

But this can't be the solution ......

Edit 3: on Showcase-Gallerys the same ... http://guillaume-hermant.piwigo.com/pic … res_photos ... is it a feature?! Why? Reason?

Last edited by WuppiGER (2016-12-15 16:55:59)

Offline

 

#2 2017-01-02 11:48:13

WuppiGER
Member
Germany
2016-05-31
38

Re: Meta: robots -> noindex,nofollow @ pictures ...

2.85: fixed ...

Reproduce on some Special-Sites:

Random-Pictures: noindex,nofollow => ok!
=> choose an picture: noindex,nofollow => ok!
=> Picture-Site: noindex,nofollow ... why? No Reason ... the canonical-URL is correct - only the Picture-URL. Yes ... Theoretical SearchEngines will not use this way (nofollow). What happend if a User posted a Link on his Website to this Random-Picture - he don't use the Canonical-URL.

recent_pics: noindex - ok (spam), but nofollow?
=> Picture-Site (picture/###/recent_pics): noindex,nofollow ... mhhh same reason like Random-Pictures: why?

recent_cats: noindex - ok, after this: no problems! ... why it is so different to recent_pics ?

best_rated: nothing ... noindex could be a good idea (spam)
=> Picture-Site: no problems ... canonical is ok, no noindex,nofollow

most_visited: exactly the same like best_rated

What is the reason for these differences? I think recent_pics, recent_cats, best_rated and most_visited should be on Main-Page only use "noindex". The Picture-Sites should be index,follow (working canonical-URLs; Therefore there should be no problems with the URLs). Random_pics: mhhh no idea. It's good for SearchEngines to get new Pic-URLs - but it could be negative?! ... mhhh. But Picture-Site: index,follow!

After using my workarounds(kill noindex,nofollow on the Picture-Site [fixed in 2.85]), Google indexed many new pages from me (nearly complete on index). Parallel on this workaround i set some pages in robots.txt to disallow (search.php! etc). More hits followed from Google!

Please do not treat the SEO topic so negatively ... SEO is so important

Thx

Offline

 

#3 2017-01-02 13:07:17

Zentalquabula
Member
2014-05-10
217

Re: Meta: robots -> noindex,nofollow @ pictures ...

Themes shouldn't mess with meta tags. Switch theme and use http://piwigo.org/ext/extension_view.php?eid=582 to customize your head elements.

I use noarchive, noodp, notranslate throughout, and then control google with robots.txt:

Code:

User-agent: googlebot
Disallow: *?*
Disallow: ?mobile
Disallow: ?lang
Disallow: /index/recent_cats
Disallow: *.mp4
Disallow: *.txt
Disallow: category$
Disallow: categories$
Disallow: /index/categories/$
Disallow: /index/categories/flat
Disallow: /picture
Disallow: /.well-known
Disallow: /apple-app
Allow: /index/category/
Allow: /index/page/
Allow: /index/contact/
Allow: */category/
Allow: /picture/*/category/
Disallow: /

User-agent: *
Disallow: /

Offline

 

#4 2017-01-09 13:54:13

WuppiGER
Member
Germany
2016-05-31
38

Re: Meta: robots -> noindex,nofollow @ pictures ...

this doesn't fix the problem ... The Main-Problem: noindex,nofollow is fixed in 2.85 ... but the other Problems ... not

=> With Head-Addon in added some code (static) into the header - yes ... and what happend with the noindex,nofollow? ...

The problem is not in the theme ... i've checked other themes - same.

I've fixed the first problem (noindex,nofollow on every Picture-Page) in my Theme - yes ... workaround ... the noindex,nofollow-Fix on every Page is gone. But random_pics, recent_pics, best_rated ...... not fixed by workaround @the moment.

Offline

 

#5 2017-01-10 15:58:46

WuppiGER
Member
Germany
2016-05-31
38

Re: Meta: robots -> noindex,nofollow @ pictures ...

Please use the google robots.txt-Tester in webmaster-Tools! Your Robots.txt isn't a good solution ..

Zentalquabula wrote:

Code:

User-agent: googlebot
Disallow: *?*
Disallow: ?mobile
Disallow: ?lang
Disallow: /index/recent_cats
Disallow: *.mp4
Disallow: *.txt
Disallow: category$
Disallow: categories$
Disallow: /index/categories/$
Disallow: /index/categories/flat
Disallow: /picture
Disallow: /.well-known
Disallow: /apple-app
Allow: /index/category/
Allow: /index/page/
Allow: /index/contact/
Allow: */category/
Allow: /picture/*/category/
Disallow: /

User-agent: *
Disallow: /

/picture/123 <- blocked by Disallow: /picture [it's your intention ... but not very good!]
/picture/123/category/ <- allowed ...
=> Problem:
The Canonical URL is /picture/123
Google crawl picture/123/category/1 ... read the canonical url picture/123 and read the robots.txt which not allowed to index picture/123 ...
Canonical URL to /picture/123 is correct - A picture can be categorized in various categories ... canonical with /category/ produces Duplicate Content. picture/123 are unique. Canonical is unique

You uses for User-agent: googlebot
Disallow: / ... an have many Disallow: /...
Why? If GOOGLE doesn't match an allow, it's disallow.

Code:

User-agent: googlebot
Allow: /...
Allow: /...
Disallow: /

should enough. Check it on webmaster-Tools

Why do you want no other crawler (bing!) on your site? with user-agent: * no one are allowed to index your page ... mhhh ... an "bad bots" don't interessted what are you written in robots.txt ;)

My robots.txt

Code:

User-agent: *
Disallow: /about.php
Disallow: /password.php
Disallow: /admin/ 
Disallow: /qsearch.php
Disallow: /index/list/
Disallow: /search.php
Disallow: /map/categories
Disallow: /identification.php
Disallow: /index/categories/created-monthly-list
Disallow: /index/categories/flat
Disallow: /.well-known
Disallow: /apple-app
Disallow: *.mp4

still at work :) not perfect.
about.php, password.php, qsearch.php, search.php, indentification.php <- no content for searchengines
/admin/ <- why? ;)
/map/categories <- Map-Plugin ... no content
/index/categories/created-monthly-list <- spam (for searchengines)
/index/categories/flat <- spam
well-known, apple-app, mp4 <- thx for this idea ;)

Most of the other "Bad"-Pages uses noindex/nofollow ...

But you have inspired me :) ... disallow: / and allow only some pages. I do not know how to interpret the other search engines this. Only a google solution and for the other searchengines: "all allowed" mhhh...?

Offline

 

#6 2017-01-10 18:32:17

Zentalquabula
Member
2014-05-10
217

Re: Meta: robots -> noindex,nofollow @ pictures ...

Google shall index the picture page in its right context, that is in the album where the picture resides. Otherwise, people will end up in one giant stream with no context and no way to get to the album.

I can see the reasons for the current scheme, namely that you can have the same picture in many albums. Then again, it is the _page_ that is interesting, not the actual image, which is indexed separately no matter what.

I really really really hate multiple URL schemes. It just explodes in indexing by bots, then explodes again in a gazillion languages. I don't want that, Google is to index just what is relevant and will not touch anything else. It will not "translate" anything, nor archive it.

Other bots can't handle such complex rules, so I tell them to go away. I'm really fed up with the evil google, who doesn't respect 410s, 404s and crawl-delays, but the others are sadly much worse.

I prefer serving humans instead of having a thousand evil bots straining my resources. I prefer having a fast site. Your mileage may vary.

That said, I still don't know from where you get your meta rules. They do not show up on my site, I only have my own headers no matter what.

Offline

 

#7 2017-05-27 18:54:08

OliverB
Member
2017-05-05
29

Re: Meta: robots -> noindex,nofollow @ pictures ...

Although this thread is somewhat old, it may be interesting for the future:

WuppiGER wrote:

<meta name="robots" content="noindex,nofollow">
[...]
This Problem isn't really new ... some months again i have the same - sometimes - not on every page-load! See: http://piwigo.org/forum/viewtopic.php?id=25710 ... But reload, reload-again doesn't work anymore ... grrrr ....

noindex and nofollow are set if the exif metadata is displayed, see image.php

Code:

if ( $metadata_showable and pwg_get_session_var('show_metadata') )
{
  $page['meta_robots']=array('noindex'=>1, 'nofollow'=>1);
}

So as long as the search bot doesn't display the EXIF data, he won't see the noindex/nofollow instructions.

Zentalquabula wrote:

Google shall index the picture page in its right context, that is in the album where the picture resides.

I strongly disagree. Google should not index the thumbnails in the album list but the larger image surrounded by informative text (Description).

Oliver

Last edited by OliverB (2017-05-27 18:56:17)

Offline

 

#8 2019-10-05 18:09:25

michaelh99
Member
Texas
2014-08-19
46

Re: Meta: robots -> noindex,nofollow @ pictures ...

OliverB wrote:

Although this thread is somewhat old, it may be interesting for the future:

noindex and nofollow are set if the exif metadata is displayed, see image.php

Code:

if ( $metadata_showable and pwg_get_session_var('show_metadata') )
{
  $page['meta_robots']=array('noindex'=>1, 'nofollow'=>1);
}

So as long as the search bot doesn't display the EXIF data, he won't see the noindex/nofollow instructions.

Zentalquabula wrote:

Google shall index the picture page in its right context, that is in the album where the picture resides.

I strongly disagree. Google should not index the thumbnails in the album list but the larger image surrounded by informative text (Description).

Oliver

Well, that's annoying as all hell.  Is that in any documentation anywhere?  I've been displaying exif data for years and had no idea that google wasn't indexing them.  I've recently been wondering how I can increase my search exposure and discovered that most of site isn't in the google index.

There's nothing private (to me) in the exif data and I want it part of the search info.

Offline

 

#9 2019-10-05 18:23:28

michaelh99
Member
Texas
2014-08-19
46

Re: Meta: robots -> noindex,nofollow @ pictures ...

Also it looks like the relevant code is in picture.php not image.php

Offline

 

Board footer

Powered by FluxBB

github twitter newsletter Donate Piwigo.org © 2002-2024 · Contact