Even if you trust App Engine to keep your app data safe, you might still want to download a local copy. Why? Because you will want to test your app locally, keeping a separate local development version.
Even if no-one uses your app yet, it's better not to always update the live version. It's just slow.
If you have a local copy of your data, you can run your app in the local app dev server and see changes immediately without deploying the app every time.
How do I get a local copy of my data?
For instance on my computer the tool got installed in
It sits conveniently in my path already, so I can just run it as "appcfg.py" from my command line without giving the full path.
That tool has several "actions" (sub-commands), as besides taking backups it is used for things like deploying the app. The relevant action here is called download_data.
How do I download my data with appcfg.py?
The action is given as the first argument to appcfg.py. In this case the relevant action is download_data.
appcfg.py download_data . --filename=data_store_backup
The following builtin needs to be enabled in your app.yaml. Add this snippet:
- remote_api: on
Running the command should now initiate a backup. Except if you run into problem with entering your login details.
Getting credentials for appcfg.py
If you got this error
oauth2client.client.ApplicationDefaultCredentialsError: The Application Default Credentials are not available. They are available if running in Google Compute Engine. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
Then you need to get a credentials file and make it visible to appcfg.py via an environment variable. However I'd suggest not following the instructions in the error message, as at least for me they just led to another error:
2017-06-15 14:23:59,331 INFO client.py:546 Attempting refresh to obtain initial access_token
2017-06-15 14:23:59,372 INFO client.py:804 Refreshing access_token
2017-06-15 14:24:01,114 INFO client.py:578 Refreshing due to a 401 (attempt 1/2)
2017-06-15 14:24:01,154 INFO client.py:804 Refreshing access_token
2017-06-15 14:24:01,614 INFO client.py:578 Refreshing due to a 401 (attempt 2/2)
2017-06-15 14:24:01,655 INFO client.py:804 Refreshing access_token
2017-06-15 14:24:02,169 INFO client.py:578 Refreshing due to a 401 (attempt 1/2)
2017-06-15 14:24:02,210 INFO client.py:804 Refreshing access_token
Instead go to IAM service accounts. Next to "Compute Engine default service account" select "Create key".
You get a file with your credentials on it. Now you can put this in your bashrc file, but to just try things out I saved it with the project and then created a "backup" script which uses it like this:
GOOGLE_APPLICATION_CREDENTIALS=compute-key.json appcfg.py download_data . --filename=data_store_backup
That sets the environment variable before calling appcfg.py. Run this script and the backup should start.
Hitting a 404?
If you get an error like the following:
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 558, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 404: Not Found
[ERROR ] Authentication Failed: Incorrect credentials or unsupported authentication type (e.g. OpenId).
It means your appcfg.py is too old.
I found this out the hard way by stepping through the files mentioned in the error message, and discovering that my appcfg.py was hitting some old ClientLogin API that is deprecated.
Installing the latest Cloud Tools version (159.0.0 as I write this) made the problem go away.