论坛 › 四川 › 黑产相关信息挖掘初探学习笔记

9q13nh 发表于 2024-8-25 11:49:48

黑产相关信息挖掘初探学习笔记

    <h2 style="color: black; text-align: left; margin-bottom: 10px;">0x01. 黑产业检测<span style="color: black;">科研</span>介绍</h2>
    <h3 style="color: black; text-align: left; margin-bottom: 10px;">黑产业介绍</h3>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">平常</span><span style="color: black;">归类</span></span>：</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">金融欺诈（如：电信诈骗）</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">贩卖服务，贩卖工具（如：为犯罪集团<span style="color: black;">供给</span>服务和工具<span style="color: black;">研发</span>等）</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;">违法</span>交易（如：非开源数据、违禁品等）</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;">特点</span>：</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">规模庞大，影响<span style="color: black;">卑劣</span>，行踪隐蔽。</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">据某2018年的<span style="color: black;">科研</span>报告<span style="color: black;">表示</span>，2017年我国黑产从业人员超过150万人（算是<span style="color: black;">有些</span>正好擦边的从业者，数量只会<span style="color: black;">更加多</span>），年产值达千亿级别。</p>
    <h3 style="color: black; text-align: left; margin-bottom: 10px;">黑产展现形式</h3>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;">Promotion Attack</span>：攻击者利用网站的弱点注入<span style="color: black;">违法</span><span style="color: black;">宣传</span>内容的攻击。（<span style="color: black;">平常</span>的<span style="color: black;">例如</span>高权重网站的留言板、search等）</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><img src="https://mmbiz.qpic.cn/mmbiz_png/5wqONuWHOttvOOIMqdogRsxXv4HnynD8Rcjda6bpRzfsiaicq3373ZsxRbhKgr1ypB6icBxN3WBtfaNyOLh6PUybw/640?wx_fmt=png&amp;tp=webp&amp;wxfrom=5&amp;wx_lazy=1&amp;wx_co=1" style="width: 50%; margin-bottom: 20px;"></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">应对思路：<span style="color: black;">经过</span>批量扫描网页页面，识别域名与内容<span style="color: black;">是不是</span>存在不一致，以<span style="color: black;">跟踪</span>攻击者。</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;">BulletProof Hosting(BPH)</span>：简<span style="color: black;">叫作</span>“防弹主机”服务，为犯罪份子<span style="color: black;">供给</span>了可抵抗<span style="color: black;">违法</span>活动投诉的技术<span style="color: black;">基本</span>架构。</p><img src="https://mmbiz.qpic.cn/mmbiz_png/5wqONuWHOttvOOIMqdogRsxXv4HnynD8HqbKvEwzWmAN5AC4296jqLl815Lo9Vbs2YPyIoacG87jZUOCiabUVnw/640?wx_fmt=png&amp;tp=webp&amp;wxfrom=5&amp;wx_lazy=1&amp;wx_co=1" style="width: 50%; margin-bottom: 20px;">
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><span style="color: black;">例如</span><span style="color: black;">供给</span>黑产网站域名和主机服务，使其能绕过运营商的限制，发送垃圾邮件和垃圾信息，<span style="color: black;">或</span>组建钓鱼网站、组件僵尸网络等。（当然，我国<span style="color: black;">由于</span>有备案制度限制，<span style="color: black;">因此</span>在国内发布垃圾<span style="color: black;">信息</span>的黑产网站基本溯源后IP都分布在境外或国内某地区）</span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">应对思路：提取BPH的动态特征，进行特征提取。</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;">Black Hat SEO</span>：黒帽搜索引擎优化，这个是最广为人知的黑产形式之一。<span style="color: black;">平常</span>技术有：隐形文本链接，关键词填充，链接工厂，webshell挂暗链等。</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">相对较新的黑帽优化手段是<span style="color: black;">蜘蛛池(Spider Pool)</span>（PS：其实<span style="color: black;">亦</span>流行好几年了）。</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><img src="https://mmbiz.qpic.cn/mmbiz_png/5wqONuWHOttvOOIMqdogRsxXv4HnynD8u2sDyqHV1YTT5ZqxXKtJG12eibjibWicnyO6ONnYxylj8UicdibN4iaw5Sxg/640?wx_fmt=png&amp;tp=webp&amp;wxfrom=5&amp;wx_lazy=1&amp;wx_co=1" style="width: 50%; margin-bottom: 20px;"></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">图上是搜索引擎蜘蛛进入网站后，网站<span style="color: black;">持续</span>动态生成一堆子域名和页面，让搜索引擎蜘蛛在站群中<span style="color: black;">始终</span>爬来爬去，<span style="color: black;">没法</span>绕出，以此<span style="color: black;">加强</span><span style="color: black;">目的</span>页面的收录率和排名。</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">更通俗的解释是：蜘蛛池<span style="color: black;">便是</span>一堆由垃圾域名<span style="color: black;">构成</span>的站群，在<span style="color: black;">每一个</span>站点下都生成海量页面（抓一堆文本内容相互组合），页面模板与正常网页没多大区别。<span style="color: black;">因为</span><span style="color: black;">每一个</span>站点都有海量的页面，<span style="color: black;">因此</span><span style="color: black;">全部</span>站群的蜘蛛总抓取量<span style="color: black;">亦</span>是巨大的。给未收录的页面引蜘蛛，<span style="color: black;">例如</span>在站群正常网页的模板中单独开一个DIV，放上未收录网页的链接（可用程序<span style="color: black;">掌控</span>，已收录的链接就<span style="color: black;">再也不</span>展示于此，只<span style="color: black;">持续</span>堆积<span style="color: black;">无</span>收录的链接），服务器不设缓存，使得蜘蛛每次<span style="color: black;">拜访</span>这块DIV中展现的链接都是不<span style="color: black;">同样</span>的。</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">蜘蛛池给<span style="color: black;">哪些</span>未收录的页面，在<span style="color: black;">短期</span>内<span style="color: black;">供给</span><span style="color: black;">海量</span>的真实的外链，是页面入口<span style="color: black;">揭发</span>多了，被抓取几率就<span style="color: black;">增多</span>，收录率自然会<span style="color: black;">提高</span>，又<span style="color: black;">由于</span>是外链，<span style="color: black;">因此</span>在排名上<span style="color: black;">亦</span>有<span style="color: black;">必定</span>的正向加分。</p>

    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;">丝绸之路(Silk Road)</span>：和<span style="color: black;">咱们</span>历史课学的那个名词<span style="color: black;">无</span>任何关系。是一个在线<span style="color: black;">鬼市</span>网站，首个被世人所知的现代暗网市场，曾经是著名的<span style="color: black;">违法</span>交易平台，<span style="color: black;">重点</span><span style="color: black;">运用</span>比特币进行交易（现已被FBI端掉）。</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;">基于黑词进行交易</span>：<span style="color: black;">经过</span><span style="color: black;">运用</span>“黑词”(<span style="color: black;">亦</span>叫“黑话”)进行<span style="color: black;">违法</span>交易。<span style="color: black;">例如</span>贩卖枪支团伙在交易中<span style="color: black;">通常</span><span style="color: black;">叫作</span>枪支为“狗”，<span style="color: black;">叫作</span>子弹为“狗粮”。加拿大某些人以“lemon”代指大麻。</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;">什么是黑产词？</span></p><span style="color: black;">1</span><span style="color: black;">2</span><span style="color: black;">3</span><span style="color: black;">“黑产词”是<span style="color: black;">伴同</span>黑产<span style="color: black;">显现</span>的<span style="color: black;">制品</span>同义词及违法<span style="color: black;">制品</span>本身的关键词的统<span style="color: black;">叫作</span>。<span style="color: black;">违法</span>商贩和买家<span style="color: black;">经过</span>协定新的词汇<span style="color: black;">暗示</span>一种<span style="color: black;">制品</span>，以此躲避监管。</span><span style="color: black;">这些隐蔽的“黑产词”<span style="color: black;">一般</span>会扭曲常用词含义，<span style="color: black;">引起</span>“外行人”<span style="color: black;">没法</span>理解其<span style="color: black;">背面</span>的含义，上面举例的<span style="color: black;">有些</span>即是“黑产词”。</span>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;">通常</span>而言，在分析黑产交易过程中最大的困难是理解<span style="color: black;">她们</span><span style="color: black;">怎样</span>交流。<span style="color: black;">因此呢</span>，相当多的黑词是“难<span style="color: black;">发掘</span>、难理解”的，毕竟<span style="color: black;">针对</span>黑产交易者<span style="color: black;">来讲</span>，识别度越低越安全。</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;">例如</span>：</p><span style="color: black;">1</span><span style="color: black;">2</span><span style="color: black;">3</span><span style="color: black;">4</span><span style="color: black;">球板 ball board -- 球类运动的职业签赌版，属于违法犯罪<span style="color: black;">行径</span>。</span><span style="color: black;">出肉 cut meat -- 和drug<span style="color: black;">相关</span></span><span style="color: black;">咕噜咕噜 guruguru -- 和drug<span style="color: black;">相关</span></span><span style="color: black;">水烟壶 hookah -- 和drug<span style="color: black;">相关</span></span>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;">更加多</span>内容可见黑产<span style="color: black;">科研</span>（地下产业<span style="color: black;">科研</span>）笔记</p>
    <h2 style="color: black; text-align: left; margin-bottom: 10px;">0x02.黑词挖掘——基于SEO</h2>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;">那样</span><span style="color: black;">怎样</span><span style="color: black;">捉捕</span>黑产行踪呢？现今的<span style="color: black;">有些</span><span style="color: black;">设备</span>学习技术<span style="color: black;">能够</span><span style="color: black;">供给</span>不少<span style="color: black;">帮忙</span>。</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><img src="https://mmbiz.qpic.cn/mmbiz_png/5wqONuWHOttvOOIMqdogRsxXv4HnynD8m6hfGbhKy3nKSrKBN0hiaYVhpbxHMQsJnRsh5AO7uGicb0ia4ZQ8gGfuw/640?wx_fmt=png&amp;tp=webp&amp;wxfrom=5&amp;wx_lazy=1&amp;wx_co=1" style="width: 50%; margin-bottom: 20px;"></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">黑产商人将商品页面交给黑帽SEO做优化，黑帽SEO对关键词做“优化”（谐音、长尾词等）后<span style="color: black;">加强</span>黑产<span style="color: black;">制品</span>页面和黑词排名。</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;">倘若</span><span style="color: black;">经过</span>搜索能找到一小批黑词，就能以此挖掘<span style="color: black;">更加多</span>的黑词。</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><img src="https://mmbiz.qpic.cn/mmbiz_png/5wqONuWHOttvOOIMqdogRsxXv4HnynD8XI9DLbbwkHEZhSVodlwjpCSCBUzw1hrl3faxoSS6FuIpez0T2m2p0Q/640?wx_fmt=png&amp;tp=webp&amp;wxfrom=5&amp;wx_lazy=1&amp;wx_co=1" style="width: 50%; margin-bottom: 20px;"></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;">经过</span>爬取页面数据清洗和识别（如：href锚链接的文本），利用NLP<span style="color: black;">关联</span>技术，整理出真正的黑词并进行<span style="color: black;">归类</span>和分级。</p>
    <h3 style="color: black; text-align: left; margin-bottom: 10px;">数据清洗思路：</h3>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">合法的关键词<span style="color: black;">常常</span>会被黑词更长</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">搜索引擎会<span style="color: black;">帮忙</span><span style="color: black;">咱们</span>标记恶意的页面</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">利用搜索引擎<span style="color: black;">举荐</span>的<span style="color: black;">关联</span>搜索词继续挖掘黑词</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">识别核心词语，并对<span style="color: black;">类似</span>的黑词进行聚类</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><img src="https://mmbiz.qpic.cn/mmbiz_png/5wqONuWHOttvOOIMqdogRsxXv4HnynD8rFS588NYibRNUGviavl4XDFZyDuSZGuYlWy1WdrOBAsohECPSMQ45YyQ/640?wx_fmt=png&amp;tp=webp&amp;wxfrom=5&amp;wx_lazy=1&amp;wx_co=1" style="width: 50%; margin-bottom: 20px;"></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;">运用</span>NLP技术去掉无关词语，如停用词(<span style="color: black;">指的是</span>在信息检索中，为节省存储空间和<span style="color: black;">加强</span>搜索效率，在处理自然语言数据（或文本）之前或之后会自动过滤掉某些字或词，这些字或词即被<span style="color: black;">叫作</span>为Stop Words，即停用词)、地点、无<span style="color: black;">道理</span>标点符号、虚词、emoji等。</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">找到<span style="color: black;">一起</span>的子串。</p>
    <h2 style="color: black; text-align: left; margin-bottom: 10px;">0x03.黑词检测——基于语义</h2>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">部分词语在<span style="color: black;">不消</span>语境下有<span style="color: black;">区别</span>词义。如可乐在<span style="color: black;">鬼市</span>上可能代指“可卡因”，葡萄可能代指“大麻”，老鼠在黑客交易中代指“木马”。<span style="color: black;">能够</span>利用<span style="color: black;">语境分析</span>来进行识别。</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><img src="https://mmbiz.qpic.cn/mmbiz_png/5wqONuWHOttvOOIMqdogRsxXv4HnynD8llE0RicCdOyCpNNoKS05EPnjo76JQbFAA8XEqFsJjFJEd4AKxTMshAA/640?wx_fmt=png&amp;tp=webp&amp;wxfrom=5&amp;wx_lazy=1&amp;wx_co=1" style="width: 50%; margin-bottom: 20px;"></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">如上图例子中，<span style="color: black;">按照</span>红框关键词和上下文语境，<span style="color: black;">能够</span>自然理解到rat在<span style="color: black;">这儿</span>不<span style="color: black;">指的是</span>老鼠，而<span style="color: black;">指的是</span>木马程序。</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><img src="https://mmbiz.qpic.cn/mmbiz_png/5wqONuWHOttvOOIMqdogRsxXv4HnynD89tQEpnTvRldPY1iaRIHANxTShknr8kCHqA0BicvzBCBFeMtHExgdYsLw/640?wx_fmt=png&amp;tp=webp&amp;wxfrom=5&amp;wx_lazy=1&amp;wx_co=1" style="width: 50%; margin-bottom: 20px;"></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">一个用于黑词识别分析的模型架构，<span style="color: black;">运用</span>多种词库，<span style="color: black;">包括</span>关系判断模型（<span style="color: black;">因此</span>需要<span style="color: black;">运用</span><span style="color: black;">设备</span>学习的<span style="color: black;">关联</span>技术，<span style="color: black;">这儿</span><span style="color: black;">触及</span>的内容<span style="color: black;">无</span><span style="color: black;">设备</span>学习和自然语言处理<span style="color: black;">基本</span>的人理解起来还是比较困难的，<span style="color: black;">意见</span>认真阅读论文原文，最后的REF）。</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><img src="https://mmbiz.qpic.cn/mmbiz_png/5wqONuWHOttvOOIMqdogRsxXv4HnynD8sibJaCY3OKYtyJHjhV5y50Fu1BmoSykmgXkDjD1cu2sWk8d8QdqzWlA/640?wx_fmt=png&amp;tp=webp&amp;wxfrom=5&amp;wx_lazy=1&amp;wx_co=1" style="width: 50%; margin-bottom: 20px;"></p>
    <h2 style="color: black; text-align: left; margin-bottom: 10px;">拓展阅读</h2>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><img src="https://mmbiz.qpic.cn/mmbiz_png/5wqONuWHOttvOOIMqdogRsxXv4HnynD8gkl5q2yjrVRPjy6nol20hRibwYIMKZhos3FL76xyYAiaavl8wTEjQzMw/640?wx_fmt=png&amp;tp=webp&amp;wxfrom=5&amp;wx_lazy=1&amp;wx_co=1" style="width: 50%; margin-bottom: 20px;"></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">How to Learn Klingon without a Dictionary: Detection and Measurement of Black Keywords Used by the Underground Economy</p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;">Reading Thieves’ Cant: Automatically Identifying and Understanding Dark Jargons from Cybercrime Marketplaces</p><span style="color: black;">转载：https:<span style="color: black;">//blog.fullstackpentest.com/black-hat-seo-preliminary-study.html</span></span><span style="color: black;">作者：黑帽SEO&nbsp;</span><span style="color: black;">欢迎<span style="color: black;">大众</span>去关注作者</span>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><strong style="color: blue;">欢迎师傅加入安全交流群(qq群：611901335)，<span style="color: black;">或</span>后台回复加群</strong></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><strong style="color: blue;"><span style="color: black;">倘若</span>想和我<span style="color: black;">一块</span>讨论，欢迎加入我的知识星球！！！</strong></span></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><img src="https://mmbiz.qpic.cn/mmbiz_png/5wqONuWHOtsasqDggsxA2mAedSic7tgVKy1GQOzcyYUsMicrcDP5BpJHC6u77RJRdnlS3oU0zkLMibaXyvcmOvH0Q/640?wx_fmt=png&amp;tp=webp&amp;wxfrom=5&amp;wx_lazy=1&amp;wx_co=1" style="width: 50%; margin-bottom: 20px;"></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><strong style="color: blue;"><span style="color: black;">扫描下图加入freebuf知识大陆</span></strong></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><img src="https://mmbiz.qpic.cn/mmbiz_png/5wqONuWHOtsasqDggsxA2mAedSic7tgVKiaib2BXNkF4STGwIkuNpaI72l29yxLq4mIn2AusjmEMUE6ibrpct8U4Vg/640?wx_fmt=png&amp;tp=webp&amp;wxfrom=5&amp;wx_lazy=1&amp;wx_co=1" style="width: 50%; margin-bottom: 20px;"></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><strong style="color: blue;"><span style="color: black;">师傅们点赞、转发、在看<span style="color: black;">便是</span>最大的支持</span></strong></p>
    <p style="font-size: 16px; color: black; line-height: 40px; text-align: left; margin-bottom: 15px;"><span style="color: black;"><strong style="color: blue;">后台回复知识星球<span style="color: black;">或</span>知识大陆<span style="color: black;">亦</span>可获取加入链接(两个加其一<span style="color: black;">就可</span>)</strong></span></p>

jm2020 发表于 2024-9-1 07:09:34

对于这个问题，我有不同的看法...

情迷布拉格 发表于 2024-9-2 14:44:58

你的见解真是独到，让我受益匪浅。

j8typz 发表于 2024-10-8 00:28:54

真情实感，其含义为认真了、走心了的意思，是如今的饭圈常用语。

nykek5i 发表于 2024-10-15 19:05:02

这篇文章真的让我受益匪浅，外链发布感谢分享！

查看完整版本: 黑产相关信息挖掘初探学习笔记