Support #8122

Meta data robots twice

Added by Samuel Quevauvillers 5 months ago. Updated 30 days ago.

Target version:
Start date:
Due date:
% Done:


Estimated time:


I have found the 2 topic abouts Metadata header tags.

I have set the meta tag for robots to "index, follow" in the wt_config.xml, but I always get a second meta tag in my header
<meta name="robots" content="noindex, nofollow" />

I get this as final :

  <meta name="robots" content="all"/>
  <meta name="description" lang="fr" content="Forestimator: outils cartographiques pour la gestion des forĂȘts Wallonnes">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <meta name="robots" content="noindex, nofollow">

How could I get rid of it ?
Many thanks


Updated by Korneel Dumon 5 months ago

  • Status changed from New to Resolved

As the second link says, Wt will only add this second header when it does not recognize your browser as a bot. This can be a bit tricky to test, try something like this:

curl  -H "User-Agent: Googlebot/2.1" http://localhost:8080

The list of bots for which this works is also in wt_config (under <user-agents type="bot">)


Updated by Samuel Quevauvillers 5 months ago

Ok thank you but there is still something I don't understand. How can I allow bots to index my site ?
I notice with the the curl cmd that if I run it as bot or not, I always get a <meta name="robots" content="noindex, nofollow">


Updated by Samuel Quevauvillers 5 months ago

Ok now this is working, I had to set back the bot list in my wt_config.xml :

<user-agents type="bot">
            <user-agent>.*Sogou web spider.*</user-agent>

Updated by Roel Standaert 30 days ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF