kb: expand authorized lab coverage and intel automation

2026-03-16 22:04:51 -07:00
--- a/04-server-security/README.md
+++ b/04-server-security/README.md
@@ -0,0 +1,26 @@
+# 服务器、TLS 与关联面实验
+
+> `LAB ONLY` | `AUTHORIZED TARGETS ONLY`
+
+## 范围元数据
+
+| 字段 | 内容 |
+|------|------|
+| 适用目标类型 | `lab-local`, `lab-public`, `authorized-third-party` |
+| 是否允许公网验证 | 允许，但必须限定为单主机或单服务面验证 |
+| 推荐最小化验证 | 先做指纹、证书、响应头和必要端口检查，再做更深验证 |
+| 禁止场景 | 泛互联网大范围端口枚举、无授权资产画像、影响服务可用性的 DoS 行为 |
+
+## 当前内容
+
+- 端口与服务指纹: [port-scanner.py](/Users/x/websafe/04-server-security/scanning/tools/port-scanner.py)
+- TLS 配置检查: [tls-scanner.py](/Users/x/websafe/04-server-security/tls/tools/tls-scanner.py)
+- 关联面分析: [site-scope-mapper.py](/Users/x/websafe/04-server-security/infrastructure/tools/site-scope-mapper.py)
+- 实验网关样例: [nginx-hardening.conf](/Users/x/websafe/05-defense/hardening/nginx-hardening.conf)
+
+## 建议实验路径
+
+1. 用 TLS 与响应头检查判断暴露面。
+2. 用端口扫描确认最小服务面。
+3. 用关联面分析确认同 IP、同证书和同代理边界。
+4. 将结果回填到 [资产模板](/Users/x/websafe/09-scope-and-targeting/asset-inventory-template.md) 与 [测试记录模板](/Users/x/websafe/09-scope-and-targeting/test-record-template.md)。
--- a/04-server-security/infrastructure/README.md
+++ b/04-server-security/infrastructure/README.md
@@ -0,0 +1,8 @@
+# 基础设施与关联面分析
+
+> `LAB ONLY` | `AUTHORIZED TARGETS ONLY`
+
+该目录用于自有或授权资产的同 IP、同证书、同反向代理和同服务面关联分析。核心目标是减少“只看单站点”的误判，识别共享边界带来的风险传播。
+
+- 工具入口: [site-scope-mapper.py](/Users/x/websafe/04-server-security/infrastructure/tools/site-scope-mapper.py)
+- 方法说明: [associated-site-analysis.md](/Users/x/websafe/09-scope-and-targeting/associated-site-analysis.md)
--- a/04-server-security/infrastructure/tools/README.md
+++ b/04-server-security/infrastructure/tools/README.md
@@ -0,0 +1,17 @@
+# 关联面分析工具
+
+> `LAB ONLY` | `AUTHORIZED TARGETS ONLY`
+
+## 工具元数据
+
+| 字段 | 内容 |
+|------|------|
+| 适用目标类型 | `lab-local`, `lab-public`, `authorized-third-party` |
+| 是否允许公网验证 | 允许 |
+| 所需授权前提 | 明确确认目标主机、站点或 IP 属于你方或已授权 |
+| 推荐最小化验证 | 解析 DNS、证书 SAN、响应头和标题，不做大范围扩展扫描 |
+| 禁止使用场景 | 大规模互联网枚举、无授权同 IP 资产画像、持续高频探测 |
+
+当前工具:
+
+- [site-scope-mapper.py](/Users/x/websafe/04-server-security/infrastructure/tools/site-scope-mapper.py)
--- a/04-server-security/infrastructure/tools/site-scope-mapper.py
+++ b/04-server-security/infrastructure/tools/site-scope-mapper.py
@@ -0,0 +1,278 @@
+#!/usr/bin/env python3
+# LAB ONLY
+# AUTHORIZED TARGETS ONLY
+"""
+Authorized Site Scope Mapper
+同 IP / 同证书 / 同反向代理 关联面分析工具
+
+支持:
+- 单主机 DNS 解析
+- 反向 DNS 查询
+- HTTP 响应头、标题与重定向观察
+- TLS 证书主题与 SAN 提取
+- 基于目标自身信息的关联主机汇总
+
+Usage:
+    python3 site-scope-mapper.py --target app.example.test --ack-authorized
+    python3 site-scope-mapper.py --target 203.0.113.10 --ports 80,443,8443 --json --ack-authorized
+
+授权边界:
+    - 仅用于自有资产、测试环境或已明确授权的目标
+    - 允许公网验证，但默认只围绕单个目标做最小化关联分析
+    - 不面向无授权第三方网站或泛互联网枚举
+"""
+
+import argparse
+import ipaddress
+import json
+import re
+import socket
+import ssl
+import warnings
+from dataclasses import asdict, dataclass, field
+from typing import Dict, List, Optional, Set
+
+warnings.filterwarnings("ignore", message="urllib3 v2 only supports OpenSSL")
+
+import requests
+
+
+DEFAULT_PORTS = [80, 443, 8080, 8443]
+
+
+@dataclass
+class HTTPObservation:
+    scheme: str
+    port: int
+    status_code: Optional[int] = None
+    location: Optional[str] = None
+    server: Optional[str] = None
+    title: Optional[str] = None
+    final_url: Optional[str] = None
+    error: Optional[str] = None
+
+
+@dataclass
+class TLSObservation:
+    port: int
+    subject_cn: Optional[str] = None
+    issuer_cn: Optional[str] = None
+    san: List[str] = field(default_factory=list)
+    not_before: Optional[str] = None
+    not_after: Optional[str] = None
+    error: Optional[str] = None
+
+
+def is_ip(value: str) -> bool:
+    try:
+        ipaddress.ip_address(value)
+        return True
+    except ValueError:
+        return False
+
+
+def resolve_host(target: str) -> Dict[str, List[str]]:
+    records = {"ipv4": [], "ipv6": []}
+    try:
+        infos = socket.getaddrinfo(target, None)
+    except socket.gaierror:
+        return records
+
+    for info in infos:
+        family = info[0]
+        address = info[4][0]
+        if family == socket.AF_INET and address not in records["ipv4"]:
+            records["ipv4"].append(address)
+        if family == socket.AF_INET6 and address not in records["ipv6"]:
+            records["ipv6"].append(address)
+    return records
+
+
+def reverse_dns(address: str) -> Optional[str]:
+    try:
+        host, _, _ = socket.gethostbyaddr(address)
+        return host
+    except Exception:
+        return None
+
+
+def extract_title(html: str) -> Optional[str]:
+    match = re.search(r"<title>(.*?)</title>", html, re.IGNORECASE | re.DOTALL)
+    if not match:
+        return None
+    return re.sub(r"\s+", " ", match.group(1)).strip()[:160]
+
+
+def observe_http(target: str, scheme: str, port: int, timeout: float) -> HTTPObservation:
+    url = f"{scheme}://{target}:{port}/"
+    try:
+        response = requests.get(
+            url,
+            timeout=timeout,
+            allow_redirects=False,
+            verify=False,
+            headers={"User-Agent": "websafe-site-scope-mapper/1.0"},
+        )
+        title = extract_title(response.text or "")
+        return HTTPObservation(
+            scheme=scheme,
+            port=port,
+            status_code=response.status_code,
+            location=response.headers.get("Location"),
+            server=response.headers.get("Server"),
+            title=title,
+            final_url=response.url,
+        )
+    except Exception as exc:
+        return HTTPObservation(scheme=scheme, port=port, error=str(exc))
+
+
+def observe_tls(target: str, port: int, timeout: float) -> TLSObservation:
+    context = ssl.create_default_context()
+    context.check_hostname = False
+    context.verify_mode = ssl.CERT_NONE
+
+    try:
+        with socket.create_connection((target, port), timeout=timeout) as sock:
+            with context.wrap_socket(sock, server_hostname=target) as ssock:
+                cert = ssock.getpeercert()
+        san = []
+        for item in cert.get("subjectAltName", []):
+            if len(item) == 2:
+                san.append(item[1])
+        subject = dict(x[0] for x in cert.get("subject", []))
+        issuer = dict(x[0] for x in cert.get("issuer", []))
+        return TLSObservation(
+            port=port,
+            subject_cn=subject.get("commonName"),
+            issuer_cn=issuer.get("commonName"),
+            san=san,
+            not_before=cert.get("notBefore"),
+            not_after=cert.get("notAfter"),
+        )
+    except Exception as exc:
+        return TLSObservation(port=port, error=str(exc))
+
+
+def parse_ports(value: str) -> List[int]:
+    ports: List[int] = []
+    for part in value.split(","):
+        part = part.strip()
+        if not part:
+            continue
+        port = int(part)
+        if port not in ports:
+            ports.append(port)
+    if len(ports) > 10:
+        raise ValueError("为避免扩大扫描范围，最多允许 10 个端口")
+    return ports
+
+
+def render_text(report: Dict) -> str:
+    lines = []
+    lines.append("=" * 68)
+    lines.append("Authorized Site Scope Mapper")
+    lines.append("=" * 68)
+    lines.append(f"Target: {report['target']}")
+    lines.append(f"Target Type: {report['target_type']}")
+    lines.append("")
+
+    dns_records = report["dns"]
+    lines.append("DNS:")
+    lines.append(f"  IPv4: {', '.join(dns_records['ipv4']) or '-'}")
+    lines.append(f"  IPv6: {', '.join(dns_records['ipv6']) or '-'}")
+    lines.append(f"  PTR : {', '.join(report['reverse_dns']) or '-'}")
+    lines.append("")
+
+    lines.append("HTTP Observations:")
+    for item in report["http"]:
+        lines.append(
+            f"  - {item['scheme']}:{item['port']} status={item.get('status_code') or '-'} "
+            f"server={item.get('server') or '-'} title={item.get('title') or '-'} "
+            f"location={item.get('location') or '-'}"
+        )
+    lines.append("")
+
+    lines.append("TLS Observations:")
+    for item in report["tls"]:
+        lines.append(
+            f"  - port {item['port']} subject={item.get('subject_cn') or '-'} "
+            f"issuer={item.get('issuer_cn') or '-'} SAN={len(item.get('san', []))}"
+        )
+    lines.append("")
+
+    lines.append("Related Hosts:")
+    related = report["related_hosts"]
+    if related:
+        for host in related:
+            lines.append(f"  - {host}")
+    else:
+        lines.append("  - None derived from on-target data")
+    return "\n".join(lines)
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description="Authorized Site Scope Mapper")
+    parser.add_argument("--target", required=True, help="目标主机名或 IP")
+    parser.add_argument(
+        "--ports",
+        default="80,443,8080,8443",
+        help="需要观察的端口列表，默认 80,443,8080,8443",
+    )
+    parser.add_argument("--timeout", type=float, default=4.0, help="请求超时时间")
+    parser.add_argument("--json", action="store_true", help="输出 JSON")
+    parser.add_argument(
+        "--ack-authorized",
+        action="store_true",
+        help="确认目标属于自有资产或已明确授权",
+    )
+    args = parser.parse_args()
+
+    if not args.ack_authorized:
+        parser.error("必须显式提供 --ack-authorized 以确认目标范围合法")
+
+    ports = parse_ports(args.ports)
+    target_type = "ip" if is_ip(args.target) else "hostname"
+    dns_records = resolve_host(args.target) if target_type == "hostname" else {"ipv4": [args.target], "ipv6": []}
+
+    reverse_hosts: Set[str] = set()
+    for address in dns_records["ipv4"] + dns_records["ipv6"]:
+        ptr = reverse_dns(address)
+        if ptr:
+            reverse_hosts.add(ptr)
+
+    http_results: List[HTTPObservation] = []
+    tls_results: List[TLSObservation] = []
+
+    for port in ports:
+        schemes = ["https"] if port in (443, 8443) else ["http"]
+        if port not in (80, 443):
+            schemes.append("https")
+        for scheme in schemes:
+            http_results.append(observe_http(args.target, scheme, port, args.timeout))
+        if port in (443, 8443):
+            tls_results.append(observe_tls(args.target, port, args.timeout))
+
+    related_hosts: Set[str] = set(reverse_hosts)
+    for item in tls_results:
+        related_hosts.update(host for host in item.san if host)
+
+    report = {
+        "target": args.target,
+        "target_type": target_type,
+        "dns": dns_records,
+        "reverse_dns": sorted(reverse_hosts),
+        "http": [asdict(item) for item in http_results],
+        "tls": [asdict(item) for item in tls_results],
+        "related_hosts": sorted(related_hosts),
+    }
+
+    if args.json:
+        print(json.dumps(report, indent=2, ensure_ascii=True))
+    else:
+        print(render_text(report))
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())
--- a/04-server-security/misconfiguration/README.md
+++ b/04-server-security/misconfiguration/README.md
@@ -0,0 +1,5 @@
+# 服务端错误配置实验
+
+> `LAB NOTE` | `规划中`
+
+该目录预留给默认目录列表、错误暴露、调试接口、代理信任链和配置合并问题的实验样例。当前相关内容分散在 [07-framework-security/server-software](/Users/x/websafe/07-framework-security/server-software/README.md) 与已有案例中。
--- a/04-server-security/misconfiguration/tools/README.md
+++ b/04-server-security/misconfiguration/tools/README.md
@@ -0,0 +1,5 @@
+# 错误配置工具说明
+
+> `LAB NOTE` | `规划中`
+
+该目录后续用于默认配置、目录暴露、调试接口和信任边界误配的辅助检查脚本。
--- a/04-server-security/nmap-scripts/README.md
+++ b/04-server-security/nmap-scripts/README.md
@@ -0,0 +1,5 @@
+# Nmap 脚本目录
+
+> `LAB NOTE` | `规划中`
+
+该目录预留给授权实验环境中的 NSE 脚本示例。当前不放置通用对外枚举脚本。
--- a/04-server-security/scanning/tools/port-scanner.py
+++ b/04-server-security/scanning/tools/port-scanner.py
@@ -13,6 +13,11 @@ Usage:
    python3 port-scanner.py -H 192.168.1.1 -p 1-1000
    python3 port-scanner.py -H 192.168.1.1 -p 80,443,8080
    python3 port-scanner.py -H 192.168.1.1 --top-ports 100
+
+授权边界:
+    - 仅用于自有资产、测试环境或已明确授权的目标
+    - 允许公网验证，但建议缩小到明确主机和必要端口范围
+    - 不面向无授权第三方网站或泛互联网枚举
 """

 import argparse
--- a/04-server-security/tls/tools/tls-scanner.py
+++ b/04-server-security/tls/tools/tls-scanner.py
@@ -12,6 +12,11 @@ TLS Scanner - TLS/SSL 安全配置扫描工具
 Usage:
    python3 tls-scanner.py -u https://example.com
    python3 tls-scanner.py -u example.com -p 443
+
+授权边界:
+    - 仅用于自有资产、测试环境或已明确授权的 TLS 终端
+    - 允许公网验证，但建议优先使用只读检查
+    - 不面向无授权第三方网站或泛互联网扫描
 """

 import argparse