2017年03月02日

SQLite3 小技 fetchall()

SQLiteは便利ですよねぇ。
辞書では物足りないが、いちいちclass定義するのもなんだかな、という時とか
中断データ作る時とか、色々便利。:memoryにしておけばファイルも作らなくていいから、
集計とかにもいいですよね。

エクセルなのにエクセルで集計できんレイアウトになってたりしても
python+xlrd+SQLite3でど根性集計もできる。脱線した。

で、技というか 1カラムしかselectしない時は、いちいちタプルで渡してくれなくてもいいので

c.execute("select filepath from master")
row =[str(item[0]) for item in c.fetchall()]

こんな感じでタプルを解除してしまえば、

for r in row:
  func(r)

みたいな感じで後続が楽、という話さ。

検索結果がA,B,Cとした場合、[(u'A',),(u'B',),(u'C',)] と帰ってきてしまう(タプルの配列になる)ので、
['A','B','C']と入れ替えてるだけです。(タプル外して、unicodeも解除)

fetchallできないような巨大な応答とかはコツコツカーソルを動かして頑張るしかないですが。


参考リンク

続きを読む
posted by koteitan at 14:12| Comment(0) | python | このブログの読者になる | 更新情報をチェックする

CVE-2016-8743  Apache HTTP Request Parsing Whitespace Defects

リクエスト中にあるCRをスペースと解釈することを悪用してスマグリング攻撃をしかけたり、とかの問題。
歴史的な問題から、HTTPリクエストの解釈はウルトラふんわりしなきゃならなかった。
HTTP/1.1もhttp/1.1でもOKにするとか、スペースの数とかね。

もっと詳しくはRFCを熟読してもらうとして、(https://tools.ietf.org/html/rfc7230#section-3.1.1
この解釈をきっちり行うことで、脆弱性を防ぐ、というわけだね。


     Apache HTTP Server, prior to release 2.4.25, accepted a broad pattern of unusual whitespace patterns from the user-agent, including bare CR, FF, VTAB in parsing the request line and request header lines, as well as HTAB in parsing the request line. Any bare CR present in request lines was treated as whitespace and remained in the request field member "the_request", while a bare CR in the request header field name would be honored as whitespace, and a bare CR in the request header field value was retained the input headers array. Implied additional whitespace was accepted in the request line and prior to the ':' delimiter of any request header lines.

    RFC7230 Section 3.5 calls out some of these whitespace exceptions, and section 3.2.3 eliminated and clarified the role of implied whitespace in the grammer of this specification. Section 3.1.1 requires exactly one single SP between the method and request-target, and between the request-target and HTTP-version, followed immediately by a CRLF sequence. None of these fields permit any (unencoded) CTL character whatsoever. Section 3.2.4 explicitly disallowed any whitespace from the request header field prior to the ':' character, while Section 3.2 disallows all CTL characters in the request header line other than the HTAB character as whitespace.

    These defects represent a security concern when httpd is participating in any chain of proxies or interacting with back-end application servers, either through mod_proxy or using conventional CGI mechanisms. In each case where one agent accepts such CTL characters and does not treat them as whitespace, there is the possiblity in a proxy chain of generating two responses from a server behind the uncautious proxy agent. In a sequence of two requests, this results in request A to the first proxy being interpreted as requests A + A' by the backend server, and if requests A and B were submitted to the first proxy in a keepalive connection, the proxy may interpret response A' as the response to request B, polluting the cache or potentially serving the A' content to a different downstream user-agent.

    These defects are addressed with the release of Apache HTTP Server 2.4.25 and coordinated by a new directive;

        HttpProtocolOptions Strict

    which is the default behavior of 2.4.25 and later. By toggling from 'Strict' behavior to 'Unsafe' behavior, some of the restrictions may be relaxed to allow some invalid HTTP/1.1 clients to communicate with the server, but this will reintroduce the possibility of the problems described in this assessment. Note that relaxing the behavior to 'Unsafe' will still not permit raw CTLs other than HTAB (where permitted), but will allow other RFC requirements to not be enforced, such as exactly two SP characters in the request line.

    Acknowledgements: We would like to thank David Dennerline at IBM Security's X-Force Researchers as well as Régis Leroy for each reporting this issue.
    Reported to security team: 10th February 2016
    Issue public: 20th December 2016
    Update Released: 20th December 2016
    Affects: 2.4.23, 2.4.20, 2.4.18, 2.4.17, 2.4.16, 2.4.12, 2.4.10, 2.4.9, 2.4.7, 2.4.6, 2.4.4, 2.4.3, 2.4.2, 2.4.1, 2.2.31, 2.2.29, 2.2.27, 2.2.26, 2.2.25, 2.2.24, 2.2.23, 2.2.22, 2.2.21, 2.2.20, 2.2.19, 2.2.18, 2.2.17, 2.2.16, 2.2.15, 2.2.14, 2.2.13, 2.2.12, 2.2.11, 2.2.10, 2.2.9, 2.2.8, 2.2.6, 2.2.5, 2.2.4, 2.2.3, 2.2.2, 2.2.0



Security risks of Unsafe

Users are strongly cautioned against toggling the Unsafe mode of operation, particularly on outward-facing, publicly accessible server deployments. If an interface is required for faulty monitoring or other custom service consumers running on an intranet, users should toggle the Unsafe option only on a specific virtual host configured to service their internal private network.
 HttpProtocolOptions Unsafe で元通りの解釈になるようなので、この修正を適用したらアプリがエラー吐きまくりでござる、\(^o^)/
 になったら、十分に安全を確認(悪意が入らないかどうかを精査)した上で、解除するってのも有りなんだろうけど。
 ま、自己責任で。セキュリティの観点からいえばアプリを直せ、になるからねぇ。
続きを読む
posted by koteitan at 13:44| Comment(0) | TrackBack(0) | 情報処理 | このブログの読者になる | 更新情報をチェックする