mirrors
/
Tasmota
mirror of https://github.com/arendst/Tasmota.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #4352 from curzon01/development 

decode-config.py: add/fix Tasmota cmnd output and filename macros
This commit is contained in:
Theo Arends 2018-11-14 08:56:30 +01:00 committed by GitHub
parent a01f9bdda1 0ff2f9072d
commit 5b5b0b928f
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 177 additions and 98 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

26
tools/decode-config.html
View File

@ -4,7 +4,7 @@
<ul>
<li><em>decode-config.py</em> uses human readable and editable <a href="http://www.json.org/">JSON</a>-format for backup/restore,</li>
<li><em>decode-config.py</em> can restore previous backuped and changed <a href="http://www.json.org/">JSON</a>-format files,</li>
<li><em>decode-config.py</em> is able to create Tasomta commands based on given configuration</li>
<li><em>decode-config.py</em> is able to create Tasmota commands based on given configuration</li>
</ul>
<p>Comparing backup files created by <em>decode-config.py</em> and *.dmp files created by Tasmota &quot;Backup/Restore Configuration&quot;: </p>
<table>
@ -69,6 +69,7 @@
<li><a href="decode-config.md#use-batch-processing">Use batch processing</a></li>
</ul>
</li>
<li><a href="decode-config.md#notes">Notes</a></li>
</ul>
</li>
</ul>
@ -190,7 +191,7 @@
<span class="hljs-selector-tag">WifiConfig</span> 5
</code></pre><p>Note: A few very specific module commands like MPC230xx, KNX and some Display commands are not supported. These are still available by JSON output.</p>
<h3 id="filter-data">Filter data</h3>
<p>The huge number of Tasomta configuration data can be overstrained and confusing, so the most of the configuration data are grouped into categories. </p>
<p>The huge number of Tasmota configuration data can be overstrained and confusing, so the most of the configuration data are grouped into categories. </p>
<p>With <em>decode-config.py</em> the following categories are available: <code>Display</code>, <code>Domoticz</code>, <code>Internal</code>, <code>KNX</code>, <code>Led</code>, <code>Logging</code>, <code>MCP230xx</code>, <code>MQTT</code>, <code>Main</code>, <code>Management</code>, <code>Pow</code>, <code>Sensor</code>, <code>Serial</code>, <code>SetOption</code>, <code>SonoffRF</code>, <code>System</code>, <code>Timers</code>, <code>Wifi</code></p>
<p>These are similary to the categories on <a href="Tasmota Command Wiki">https://github.com/arendst/Sonoff-Tasmota/wiki/Commands</a>.</p>
<p>To filter outputs to a subset of groups use the <code>-g</code> or <code>--group</code> arg concatenating the grooup you want, e. g.</p>
@ -247,12 +248,16 @@
-i, <span class="hljs-comment">--restore-file &lt;filename&gt;</span>
file to restore configuration from (<span class="hljs-keyword">default</span>: <span class="hljs-type">None</span>).
<span class="hljs-type">Replacements</span>: @v=firmware version, @f=device friendly
name, @h=device hostname
<span class="hljs-type">Replacements</span>: @v=firmware version from config,
@f=device friendly name from config, @h=device
hostname from config, @<span class="hljs-type">H</span>=device hostname from device
(-d arg only)
-o, <span class="hljs-comment">--backup-file &lt;filename&gt;</span>
file to backup configuration to (<span class="hljs-keyword">default</span>: <span class="hljs-type">None</span>).
<span class="hljs-type">Replacements</span>: @v=firmware version, @f=device friendly
name, @h=device hostname
<span class="hljs-type">Replacements</span>: @v=firmware version from config,
@f=device friendly name from config, @h=device
hostname from config, @<span class="hljs-type">H</span>=device hostname from device
(-d arg only)
-t, <span class="hljs-comment">--backup-type json|bin|dmp</span>
backup filetype (<span class="hljs-keyword">default</span>: 'json')
-<span class="hljs-type">E</span>, <span class="hljs-comment">--extension append filetype extension for -i and -o filename</span>
@ -339,3 +344,12 @@ json-indent <span class="hljs-number">2</span>
</code></pre><p>or under windows</p>
<pre><code><span class="hljs-keyword">for</span> device <span class="hljs-keyword">in</span> (sonoff1 sonoff2 sonoff3) <span class="hljs-keyword">do</span> <span class="hljs-keyword">python</span> decode-config.py -c my.conf -d %device -o Config_@f_@v
</code></pre><p>will produce JSON configuration files for host sonoff1, sonoff2 and sonoff3 using friendly name and Tasmota firmware version for backup filenames.</p>
<h2 id="notes">Notes</h2>
<p>Some general notes:</p>
<ul>
<li>Filename replacement macros <strong>@h</strong> and <strong>@H</strong>:<ul>
<li><strong>@h</strong><br>The <strong>@h</strong> replacement macro uses the hostname configured with the Tasomta Wifi <code>Hostname &lt;host&gt;</code> command (defaults to <code>%s-%04d</code>). It will not use the network hostname of your device because this is not available when working with files only (e.g. <code>--file &lt;filename&gt;</code> as source).<br>To prevent having a useless % in your filename, <strong>@h</strong> will not replaced by configuration data hostname if this contains &#39;%&#39; characters.</li>
<li><strong>@H</strong><br>If you want to use the network hostname within your filename, use the <strong>@H</strong> replacement macro instead - but be aware this will only replaced if you are using a network device as source (<code>-d</code>, <code>--device</code>, <code>--host</code>); it will not work when using a file as source (<code>-f</code>, <code>--file</code>)</li>
</ul>
</li>
</ul>

26
tools/decode-config.md
View File

@ -4,7 +4,7 @@ _decode-config.py_ is able to backup and restore Sonoff-Tasmota configuration.
In contrast to the Tasmota build-in "Backup/Restore Configuration" function,
* _decode-config.py_ uses human readable and editable [JSON](http://www.json.org/)-format for backup/restore,
* _decode-config.py_ can restore previous backuped and changed [JSON](http://www.json.org/)-format files,
* _decode-config.py_ is able to create Tasomta commands based on given configuration
* _decode-config.py_ is able to create Tasmota commands based on given configuration
Comparing backup files created by *decode-config.py* and *.dmp files created by Tasmota "Backup/Restore Configuration":
@ -38,6 +38,7 @@ _decode-config.py_ is able to handle Tasmota configurations for release version
* [Config file](decode-config.md#config-file)
* [Using Tasmota binary configuration files](decode-config.md#using-tasmota-binary-configuration-files)
* [Use batch processing](decode-config.md#use-batch-processing)
* [Notes](decode-config.md#notes)
## Prerequisite
* [Python](https://en.wikipedia.org/wiki/Python_(programming_language))
@ -191,7 +192,7 @@ Example:
Note: A few very specific module commands like MPC230xx, KNX and some Display commands are not supported. These are still available by JSON output.
### Filter data
The huge number of Tasomta configuration data can be overstrained and confusing, so the most of the configuration data are grouped into categories.
The huge number of Tasmota configuration data can be overstrained and confusing, so the most of the configuration data are grouped into categories.
With _decode-config.py_ the following categories are available: `Display`, `Domoticz`, `Internal`, `KNX`, `Led`, `Logging`, `MCP230xx`, `MQTT`, `Main`, `Management`, `Pow`, `Sensor`, `Serial`, `SetOption`, `SonoffRF`, `System`, `Timers`, `Wifi`
@ -266,12 +267,16 @@ For advanced help use `-H` or `--full-help`:
-i, --restore-file <filename>
file to restore configuration from (default: None).
Replacements: @v=firmware version, @f=device friendly
name, @h=device hostname
Replacements: @v=firmware version from config,
@f=device friendly name from config, @h=device
hostname from config, @H=device hostname from device
(-d arg only)
-o, --backup-file <filename>
file to backup configuration to (default: None).
Replacements: @v=firmware version, @f=device friendly
name, @h=device hostname
Replacements: @v=firmware version from config,
@f=device friendly name from config, @h=device
hostname from config, @H=device hostname from device
(-d arg only)
-t, --backup-type json|bin|dmp
backup filetype (default: 'json')
-E, --extension append filetype extension for -i and -o filename
@ -374,3 +379,12 @@ or under windows
for device in (sonoff1 sonoff2 sonoff3) do python decode-config.py -c my.conf -d %device -o Config_@f_@v
will produce JSON configuration files for host sonoff1, sonoff2 and sonoff3 using friendly name and Tasmota firmware version for backup filenames.
## Notes
Some general notes:
* Filename replacement macros **@h** and **@H**:
* **@h**
The **@h** replacement macro uses the hostname configured with the Tasomta Wifi `Hostname <host>` command (defaults to `%s-%04d`). It will not use the network hostname of your device because this is not available when working with files only (e.g. `--file <filename>` as source).
To prevent having a useless % in your filename, **@h** will not replaced by configuration data hostname if this contains '%' characters.
* **@H**
If you want to use the network hostname within your filename, use the **@H** replacement macro instead - but be aware this will only replaced if you are using a network device as source (`-d`, `--device`, `--host`); it will not work when using a file as source (`-f`, `--file`)

223
tools/decode-config.py
View File

@ -1,13 +1,13 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
VER = '2.1.0006'
VER = '2.1.0007'
"""
decode-config.py - Backup/Restore Sonoff-Tasmota configuration data
Copyright (C) 2018 Norbert Richter <nr@prsolution.eu>
This program is free software: you can redistribute it and/or modfy
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
@ -73,12 +73,16 @@ Usage: decode-config.py [-f <filename>] [-d <host>] [-P <port>]
-i, --restore-file <filename>
file to restore configuration from (default: None).
Replacements: @v=firmware version, @f=device friendly
name, @h=device hostname
Replacements: @v=firmware version from config,
@f=device friendly name from config, @h=device
hostname from config, @H=device hostname from device
(-d arg only)
-o, --backup-file <filename>
file to backup configuration to (default: None).
Replacements: @v=firmware version, @f=device friendly
name, @h=device hostname
Replacements: @v=firmware version from config,
@f=device friendly name from config, @h=device
hostname from config, @H=device hostname from device
(-d arg only)
-t, --backup-type json|bin|dmp
backup filetype (default: 'json')
-E, --extension append filetype extension for -i and -o filename
@ -461,7 +465,7 @@ Setting_5_10_0 = {
'altitude': ('<h', 0x2F6, (None, '-30000 <= $ <= 30000', ('Sensor', '"Altitude {}".format($)')) ),
'tele_period': ('<H', 0x2F8, (None, '0 <= $ <= 1 or 10 <= $ <= 3600',('MQTT', '"TelePeriod {}".format($)')) ),
'ledstate': ('B', 0x2FB, (None, '0 <= ($ & 0x7) <= 7', ('Main', '"LedState {}".format($)')) ),
'param': ('B', 0x2FC, ([23], None, ('SetOption', '"SetOption{} {}".format(#+32,$)')) ),
'param': ('B', 0x2FC, ([23], None, ('SetOption', '"SetOption{} {}".format(#+31,$)')) ),
'state_text': ('11s', 0x313, ([4], None, ('MQTT', '"StateText{} {}".format(#,$)')) ),
'domoticz_update_timer': ('<H', 0x340, (None, '0 <= $ <= 3600', ('Domoticz', '"DomoticzUpdateTimer {}".format($)')) ),
'pwm_range': ('<H', 0x342, (None, '$==1 or 255 <= $ <= 1023', ('Management', '"PwmRange {}".format($)')) ),
@ -615,7 +619,7 @@ Setting_5_14_0.update ({
'dow': ('<H', (0x2E2,3, 8), (None, '1 <= $ <= 7', ('Management', None)) ),
'hour': ('<H', (0x2E2,5,11), (None, '0 <= $ <= 23', ('Management', None)) ),
}, 0x2E2, ([2], None, ('Management', None)), (None, False) ),
'param': ('B', 0x2FC, ([18], None, ('SetOption', '"SetOption{} {}".format(#+32,$)')) ),
'param': ('B', 0x2FC, ([18], None, ('SetOption', '"SetOption{} {}".format(#+31,$)')) ),
'toffset': ('<h', 0x30E, ([2], None, ('Management', '"{cmnd} {hemis},{week},{month},{dow},{hour},{toffset}".format(cmnd="TimeSTD" if idx==1 else "TimeDST", hemis=@["tflag"][#-1]["hemis"], week=@["tflag"][#-1]["week"], month=@["tflag"][#-1]["month"], dow=@["tflag"][#-1]["dow"], hour=@["tflag"][#-1]["hour"], toffset=value)')) ),
})
# ======================================================================
@ -692,8 +696,7 @@ Setting_6_2_1['flag2'][0].update ({
# ======================================================================
Setting_6_2_1_2 = copy.deepcopy(Setting_6_2_1)
Setting_6_2_1_2['flag3'][0].update ({
# hardcoded to 0
'user_esp8285_enable': ('<L', (0x3A0,1, 1), (None, None, ('System', None)) ),
'user_esp8285_enable': ('<L', (0x3A0,1, 1), (None, None, ('SetOption', '"SetOption51 {}".format($)')) ),
})
# ======================================================================
Setting_6_2_1_3 = copy.deepcopy(Setting_6_2_1_2)
@ -732,7 +735,7 @@ Setting_6_2_1_19.update({
})
Setting_6_2_1_20 = Setting_6_2_1_19
Setting_6_2_1_20['flag3'][0].update ({
'gui_hostname_ip': ('<L', (0x3A0,1,3), (None, None, ('System', None)) ),
'gui_hostname_ip': ('<L', (0x3A0,1,3), (None, None, ('SetOption', '"SetOption53 {}".format($)')) ),
})
# ======================================================================
Setting_6_3_0 = copy.deepcopy(Setting_6_2_1_20)
@ -757,7 +760,7 @@ Setting_6_3_0_4.update({
'displays': ('<L', 0x7B0, (None, None, ('System', None)), '"0x{:08x}".format($)' ),
})
Setting_6_3_0_4['flag3'][0].update ({
'tuya_apply_o20': ('<L', (0x3A0,1, 4), (None, None, ('System', None)) ),
'tuya_apply_o20': ('<L', (0x3A0,1, 4), (None, None, ('SetOption', '"SetOption54 {}".format($)')) ),
})
# ======================================================================
Settings = [
@ -1091,11 +1094,13 @@ def MakeFilename(filename, filetype, configmapping):
@param filename:
original filename possible containing replacements:
@v:
Tasmota version
Tasmota version from config data
@f:
friendlyname
friendlyname from config data
@h:
hostname
hostname from config data
@H:
hostname from device (-d arg only)
@param filetype:
FileType.x object - creates extension if not None
@param configmapping:
@ -1104,14 +1109,25 @@ def MakeFilename(filename, filetype, configmapping):
@return:
New filename with replacements
"""
v = f1 = f2 = f3 = f4 = ''
config_version = config_friendlyname = config_hostname = device_hostname = ''
if 'version' in configmapping:
v = GetVersionStr( int(str(configmapping['version']), 0) )
filename = filename.replace('@v', v)
config_version = GetVersionStr( int(str(configmapping['version']), 0) )
if 'friendlyname' in configmapping:
filename = filename.replace('@f', configmapping['friendlyname'][0] )
config_friendlyname = configmapping['friendlyname'][0]
if 'hostname' in configmapping:
filename = filename.replace('@h', configmapping['hostname'] )
if configmapping['hostname'].find('%') < 0:
config_hostname = configmapping['hostname']
if filename.find('@H') >= 0 and args.device is not None:
device_hostname = GetTasmotaHostname(args.device, args.port, username=args.username, password=args.password)
if device_hostname is None:
device_hostname = ''
filename = filename.replace('@v', config_version)
filename = filename.replace('@f', config_friendlyname )
filename = filename.replace('@h', config_hostname )
filename = filename.replace('@H', device_hostname )
dirname = basename = ext = ''
name = filename
@ -1196,6 +1212,94 @@ def LoadTasmotaConfig(filename):
return encode_cfg
def TasmotaGet(cmnd, host, port, username=DEFAULTS['source']['username'], password=None, contenttype = None):
"""
Tasmota http request
@param host:
hostname or IP of Tasmota device
@param port:
http port of Tasmota device
@param username:
optional username for Tasmota web login
@param password
optional password for Tasmota web login
@return:
binary config data (encrypted) or None on error
"""
body = None
# read config direct from device via http
c = pycurl.Curl()
buffer = io.BytesIO()
c.setopt(c.WRITEDATA, buffer)
header = HTTPHeader()
c.setopt(c.HEADERFUNCTION, header.store)
c.setopt(c.FOLLOWLOCATION, True)
c.setopt(c.URL, MakeUrl(host, port, cmnd))
if username is not None and password is not None:
c.setopt(c.HTTPAUTH, c.HTTPAUTH_BASIC)
c.setopt(c.USERPWD, username + ':' + password)
c.setopt(c.HTTPGET, True)
c.setopt(c.VERBOSE, False)
responsecode = 200
try:
c.perform()
responsecode = c.getinfo(c.RESPONSE_CODE)
response = header.response()
except Exception, e:
exit(e[0], e[1],line=inspect.getlineno(inspect.currentframe()))
finally:
c.close()
if responsecode >= 400:
exit(responsecode, 'HTTP result: {}'.format(header.response()),line=inspect.getlineno(inspect.currentframe()))
elif contenttype is not None and header.contenttype()!=contenttype:
exit(ExitCode.DOWNLOAD_CONFIG_ERROR, "Device did not response properly, may be Tasmota webserver admin mode is disabled (WebServer 2)",line=inspect.getlineno(inspect.currentframe()))
try:
body = buffer.getvalue()
except:
pass
return responsecode, body
def GetTasmotaHostname(host, port, username=DEFAULTS['source']['username'], password=None):
"""
Get Tasmota hostname from device
@param host:
hostname or IP of Tasmota device
@param port:
http port of Tasmota device
@param username:
optional username for Tasmota web login
@param password
optional password for Tasmota web login
@return:
Tasmota real hostname or None on error
"""
hostname = None
loginstr = ""
if password is not None:
loginstr = "user={}&password={}&".format(urllib2.quote(username), urllib2.quote(password))
# get hostname
responsecode, body = TasmotaGet("cm?{}cmnd=status%205".format(loginstr), host, port, username=username, password=password)
if body is not None:
jsonbody = json.loads(body)
if "StatusNET" in jsonbody and "Hostname" in jsonbody["StatusNET"]:
hostname = jsonbody["StatusNET"]["Hostname"]
if args.verbose:
message("Hostname for '{}' retrieved: '{}'".format(host, hostname), typ=LogType.INFO)
return hostname
def PullTasmotaConfig(host, port, username=DEFAULTS['source']['username'], password=None):
"""
Pull config from Tasmota device
@ -1212,43 +1316,9 @@ def PullTasmotaConfig(host, port, username=DEFAULTS['source']['username'], passw
@return:
binary config data (encrypted) or None on error
"""
responsecode, body = TasmotaGet('dl', host, port, username, password, contenttype='application/octet-stream')
encode_cfg = None
# read config direct from device via http
c = pycurl.Curl()
buffer = io.BytesIO()
c.setopt(c.WRITEDATA, buffer)
header = HTTPHeader()
c.setopt(c.HEADERFUNCTION, header.store)
c.setopt(c.FOLLOWLOCATION, True)
c.setopt(c.URL, MakeUrl(host, port, 'dl'))
if username is not None and password is not None:
c.setopt(c.HTTPAUTH, c.HTTPAUTH_BASIC)
c.setopt(c.USERPWD, username + ':' + password)
c.setopt(c.VERBOSE, False)
responsecode = 200
try:
c.perform()
responsecode = c.getinfo(c.RESPONSE_CODE)
response = header.response()
except Exception, e:
exit(e[0], e[1],line=inspect.getlineno(inspect.currentframe()))
finally:
c.close()
if responsecode >= 400:
exit(responsecode, 'HTTP result: {}'.format(header.response()),line=inspect.getlineno(inspect.currentframe()))
elif header.contenttype()!='application/octet-stream':
exit(ExitCode.DOWNLOAD_CONFIG_ERROR, "Device did not response properly, may be Tasmota webserver admin mode is disabled (WebServer 2)",line=inspect.getlineno(inspect.currentframe()))
try:
encode_cfg = buffer.getvalue()
except:
pass
return encode_cfg
return body
def PushTasmotaConfig(encode_cfg, host, port, username=DEFAULTS['source']['username'], password=None):
@ -1273,40 +1343,21 @@ def PushTasmotaConfig(encode_cfg, host, port, username=DEFAULTS['source']['usern
if isinstance(encode_cfg, bytearray):
encode_cfg = str(encode_cfg)
c = pycurl.Curl()
buffer = io.BytesIO()
c.setopt(c.WRITEDATA, buffer)
header = HTTPHeader()
c.setopt(c.HEADERFUNCTION, header.store)
c.setopt(c.FOLLOWLOCATION, True)
# get restore config page first to set internal Tasmota vars
c.setopt(c.URL, MakeUrl(host, port, 'rs?'))
if args.username is not None and args.password is not None:
c.setopt(c.HTTPAUTH, c.HTTPAUTH_BASIC)
c.setopt(c.USERPWD, args.username + ':' + args.password)
c.setopt(c.HTTPGET, True)
c.setopt(c.VERBOSE, False)
responsecode = 200
try:
c.perform()
responsecode = c.getinfo(c.RESPONSE_CODE)
except Exception, e:
c.close()
return e[0], e[1]
if responsecode >= 400:
c.close()
return responsecode, header.response()
elif header.contenttype() != 'text/html':
c.close()
return ExitCode.UPLOAD_CONFIG_ERROR, "Device did not response properly, may be Tasmota webserver admin mode is disabled (WebServer 2)"
responsecode, body = TasmotaGet('rs?', host, port, username, password, contenttype='text/html')
if body is None:
return responsecode, "ERROR"
# post data
header.clear()
c = pycurl.Curl()
header = HTTPHeader()
c.setopt(c.HEADERFUNCTION, header.store)
c.setopt(c.WRITEFUNCTION, lambda x: None)
c.setopt(c.POST, 1)
c.setopt(c.URL, MakeUrl(host, port, 'u2'))
if username is not None and password is not None:
c.setopt(c.HTTPAUTH, c.HTTPAUTH_BASIC)
c.setopt(c.USERPWD, username + ':' + password)
try:
isfile = os.path.isfile(encode_cfg)
except:
@ -2501,12 +2552,12 @@ def ParseArgs():
metavar='<filename>',
dest='restorefile',
default=DEFAULTS['backup']['backupfile'],
help="file to restore configuration from (default: {}). Replacements: @v=firmware version, @f=device friendly name, @h=device hostname".format(DEFAULTS['backup']['restorefile']))
help="file to restore configuration from (default: {}). Replacements: @v=firmware version from config, @f=device friendly name from config, @h=device hostname from config, @H=device hostname from device (-d arg only)".format(DEFAULTS['backup']['restorefile']))
backup.add_argument('-o', '--backup-file',
metavar='<filename>',
dest='backupfile',
default=DEFAULTS['backup']['backupfile'],
help="file to backup configuration to (default: {}). Replacements: @v=firmware version, @f=device friendly name, @h=device hostname".format(DEFAULTS['backup']['backupfile']))
help="file to backup configuration to (default: {}). Replacements: @v=firmware version from config, @f=device friendly name from config, @h=device hostname from config, @H=device hostname from device (-d arg only)".format(DEFAULTS['backup']['backupfile']))
backup_file_formats = ['json', 'bin', 'dmp']
backup.add_argument('-t', '--backup-type',
metavar='|'.join(backup_file_formats),