The curl and the wget commands make it easy to download content from web sites.

What does this mean? Technically, Google does not have to index the content, these words, the HTML, the images, etc on this page fo the page to be in Google's index.