AI Agent 这个概念最近被炒得很热，从管理日程的语音助手到仓库里跑来跑去的机器人，大家都在谈论 Agent 的"自主性"。但是真正让 Agent 变得 intelligent 的核心技术，其实是强化学习（Reinforcement Learning, RL）。

想象一下自动驾驶汽车在复杂路况中的决策，或者量化交易系统在市场波动时的操作——这些场景的共同点是什么？环境动态变化，规则无法穷尽。传统的 if-else 逻辑在这里完全失效，而 RL 恰好擅长处理这类问题。它让 Agent 像人一样，从试错中学习，在探索（exploration）和利用（exploitation）之间找平衡。

本文会从 RL 的数学基础讲起，然后深入到知识图谱的多跳推理，最后在 LangGraph 框架里搭建一个 RL 驱动的智能系统。

RL 的底层逻辑：马尔可夫决策过程教机器人玩游戏的过程其实很像教小孩骑自行车——没人会给它写一本"如何骑车的完整手册"，而是让它自己试，摔了就爬起来再试，在练习的同时我们给予帮助，如果做对就鼓励，做错了给个提示应该怎么做，这样慢慢他就学会了，而 RL 的数学框架就是根据这个流程设计的。

RL 的核心是 Markov Decision Process（MDP），它把决策过程拆成几个要素：状态（State）、动作（Action）、转移概率（Transition）、奖励（Reward）和折扣因子（Discount Factor）。形式化表示是个五元组 (S, A, P, R, γ)。

State（状态集合 S）就是"现在是什么情况"。比如对于一个走迷宫的机器人，状态可能就是它在网格上的坐标（3,5）；而对于自动驾驶系统，状态会复杂得多——车速、位置、周围障碍物、天气情况等等都得考虑进去。状态可以是离散的（grid position），也可以是连续的（speed from 0 to 100 mph）。

Action（动作集合 A）知道自己的状态后，下一步能干什么？走迷宫的机器人只有上下左右四个选择，但一个优化仓库作业的物流 Agent 可能要在"取包裹 A"、"送到货架 B"、"等待库存更新"等等决策之间做选择。动作空间的设计直接影响学习难度。

Transition Probability（转移概率 P(s'|s,a)）这个概念很关键但经常被忽略。在理想状态下，机器人"向右走"就一定会到右边的格子；但现实世界有噪音——地面可能很滑，有 80%概率到达目标位置，10%概率滑到上面，10%滑到下面。P 函数描述的就是这种不确定性，输出的概率总和为 1。没有这个，RL 根本应付不了真实环境的复杂度。

Reward（奖励函数 R(s,a,s')）Reward 是个标量，可能是+10 或-5，Agent 通过它就可以知道刚当前操作后的好坏。比如取到球了？+1。撞墙了？-1。商业场景里奖励设计更直接——准时送达+50，燃油浪费-20。

不过奖励设计是个大坑。太稀疏（只有最终结果有反馈）会让 Agent 瞎转悠找不到方向；太密集（每一步都给分）又容易让它钻空子，比如游戏 AI 可能会发现原地打转能刷小分，就跟游戏里面我们找到漏洞后刷分一样，AI 也会刷份。

Discount Factor（折扣因子 γ）γ取值在[0,1]之间，决定了 Agent 有多"远视"。γ=0 的话 Agent 只看眼前利益，γ=1 则会无限规划未来（计算量爆炸）。实际应用中通常设 0.9 或 0.99，比如股票交易系统既要抓短期机会，又不能为了眼前收益把整个 portfolio 搞崩。

这五个要素支撑起了 Q-Learning、SARSA 这些经典算法。Q-Learning 是 off-policy 的，可以从任何动作的经验中学习；SARSA 是 on-policy 的，只从实际执行的动作学习。对 Agent 来说，Partially Observable MDP（POMDP）更贴近现实——系统必须从有限观测推断隐藏状态，就像人在信息不完整时做判断。

知识图谱的多跳推理知识图谱（Knowledge Graph）本质上是个大网络，节点代表实体（entity），边代表关系（relation）。"巴黎"通过"capital_of"连到"法国"，"法国"通过"located_in"连到"欧洲"——这种结构让 AI 可以做复杂的关联查询。

但是问题就来了：面对"埃菲尔铁塔所在国家的首都人口是多少"这种 query，Agent 需要跳好几步——从 Eiffel Tower → located_in → France → capital → Paris → population。路径很多，怎么选最优的？如果暴力遍历，那么在大规模 KG 上根本跑不动。

RL 就来了，它把图遍历建模成 MDP：当前节点是 state，选择哪条边是 action，到达目标节点给正 reward，走错路给负 reward。Agent 经过训练后能学会高效路径，不用每次都试遍所有可能。

Reward shaping 在这里尤其重要。基础设定可能是"到终点+10，其他步骤 0"，但是这样太稀疏了，Agent 在大图里会迷路。更好的做法是给中间步骤也设奖励——比如每接近目标一步就+1，偏离就-0.5。关键是这些额外奖励不能改变最优策略的本质，只是引导探索方向，就像给登山者设的路标并不妨碍他发现更好的路线。

Graph Neural Network（GNN）经常和 RL 搭配使用。GNN 把每个节点的邻域信息压缩成 embedding vector，这个 vector 再喂给 RL 系统做 value prediction。本质上是把图的拓扑结构编码成数值表示，让决策更快更准。

实际应用场景很多。医疗 KG 里有疾病、症状、治疗、药物等节点，关系包括"causes"、"treats"等。RL Agent 可以从症状出发，跳转到可能的诊断，最后找到合适疗法——奖励快速准确的路径（少做不必要检查），惩罚风险路径（不推荐过时药物）。供应链网络也类似，节点是供应商和产品，边是依赖关系，RL 可以学习避开瓶颈，优化库存调度。

LangGraph 实例：把 RL 嵌入工作流 LangChain 做的是把不同 AI 组件串起来（call LLM → query database → use tool），LangGraph 更进一步，让你把 workflow 设计成有向无环图（DAG）。不是硬编码每条路径，而是把 RL Agent 嵌到节点里，让整个图变成动态决策系统。

比如云计算的资源调度场景，节点可能代表不同服务器或任务（"allocate CPU to email processing"），RL Agent 在关键节点学习"分配更多算力"或"迁移负载"这类动作，奖励基于系统运行指标（低延迟 high reward，crash 或 overload 则 penalty）。

下面我们模拟在线教育平台选课逻辑。有三个 topic（math, science, history），每个都有 urgency score 表示学生的需求程度，Agent 要选出最该教的内容。

安装依赖

pip install langgraph langchain torch gym numpy 用 Gym 框架定义练习空间。State 是三个 topic 的 urgency 向量，比如[0.7, 0.2, 0.9]——数值越高越紧迫；Action 是选择 topic 0/1/2；Reward 设计成负值，鼓励 Agent 优先选高紧迫的（最小化负 loss）；Episode 在所有 topic 都 mastered 时结束。

import gym

from gym import spaces

import numpy as np

class LearningEnv(gym.Env):

def init(self):

# Actions: Pick topic 0,1,2 to teachself.action_space = spaces.Discrete(3)

# State: Urgency levels (0 calm, 1 urgent) for three topicsself.observation_space = spaces.Box(low=0, high=1, shape=(3,))

# Start random, like a new student's needsself.state = np.random.rand(3)

def step(self, action):      # Reward: negative scaled by urgency (high urgency = less negative = better)    reward = -self.state[action] * 10      # Update: teaching halves urgency    self.state[action] *= 0.5      # Done when all topics below 0.1    done = all(s < 0.1 for s in self.state)      return self.state, reward, done, {}    def reset(self):      self.state = np.random.rand(3)       return self.state

这里做个简单的说明：

gym.Env 提供了标准接口，有 step（执行动作获取反馈）和 reset（重新开始）

Discrete action space 让选择简单直接；Box state 允许连续值（实际数据可以 normalize 到 0-1）

Reward 的负值设计：urgency 0.9 对应-9（"还行"），0.1 对应-1（"糟糕"），Agent 会学着最小化这个 loss

每次 teaching 把 urgency 减半，模拟学习进度

reset 对 repeated training 很关键，避免数据污染

我们这里选择搭建 Actor-Critic 架构。Actor 输出动作概率分布，Critic 评估状态价值。用 PyTorch 实现，保持轻量。

import torch

import torch.nn as nn

import torch.optim as optim

class TutorPolicy(nn.Module):

def init(self, obs_size, act_size):

super().init()

# Actor: state to action probabilitiesself.actor = nn.Sequential(

nn.Linear(obs_size, 64),

nn.ReLU(),

nn.Linear(64, act_size),

nn.Softmax(dim=-1) # probabilities sum to 1)

# Critic: state to value estimateself.critic = nn.Sequential(

nn.Linear(obs_size, 64),

nn.ReLU(),

nn.Linear(64, 1) # single value output)

def forward(self, obs):      probs = self.actor(obs)      value = self.critic(obs)       return probs, value

网络结构分析：

Actor 部分：Linear 层做线性变换，ReLU 引入非线性（学习复杂模式），Softmax 保证输出是合法概率分布

Critic 部分：类似结构但输出单个 value，用来估计当前 state 的期望回报

forward()是推理过程，输入状态张量，输出 action probs 和 value estimate

这里有一个注意的地方，obs 必须是 torch.tensor 格式，否则会报错。如果看到 nan 输出，请检查输入类型。

有了环境和模型，开始训练。循环跑 episodes，根据 reward 更新参数。

def train_tutor(env, policy, optimizer, epochs=200):

for epoch in range(epochs):

obs = torch.tensor(env.reset(), dtype=torch.float32)

done = False

episode_reward = 0

while not done:

probs, value = policy(obs)

action = torch.multinomial(probs, 1).item() # sample for exploration

next_obs, reward, done, _ = env.step(action)

next_obs = torch.tensor(next_obs, dtype=torch.float32)

episode_reward += reward

        # Loss computation        policy_loss = -torch.log(probs[action]) * reward  # REINFORCE gradient        value_loss = (reward - value)**2  # TD error squared        loss = policy_loss + value_loss                  optimizer.zero_grad()          loss.backward()          optimizer.step()          obs = next_obs              if epoch % 20 == 0:          print(f"Epoch {epoch}: Reward = {episode_reward:.2f}")  

Initialize and train

env = LearningEnv()

policy = TutorPolicy(3, 3)

optimizer = optim.Adam(policy.parameters(), lr=0.001)

train_tutor(env, policy, optimizer)训练逻辑如下：

外层循环是 epoch，每次 reset 环境开始新 episode

内层 while 循环是单个 episode 的 gameplay：模型推理→采样动作（multinomial 保证 exploration）→环境反馈→记录 reward

Loss 由两部分组成：policy_loss 基于 REINFORCE 算法（-log prob * reward，放大好动作概率）；value_loss 是 TD error 的平方（拟合 value 估计）

标准的梯度下降三步走：zero_grad 清空旧梯度→backward 计算新梯度→step 更新参数

Print 观察训练效果，reward 从-40 涨到-15 说明策略在优化（负得少=选得准）

训练完成后把模型部署到 workflow 里，作为决策节点。

https://vk.com/topic-233841105_53945063

https://vk.com/topic-233841100_55144580

https://vk.com/topic-233841106_54431404

https://vk.com/topic-233841102_54731758

https://vk.com/topic-233841104_54059717

https://vk.com/topic-233841108_54550900

https://vk.com/topic-233841101_55622257

https://vk.com/topic-233841103_54357987

https://vk.com/topic-233841107_55344677

https://vk.com/topic-233841099_54771566

https://vk.com/topic-233840272_54731702

https://vk.com/topic-233840276_54431251

https://vk.com/topic-233840275_53945017

https://vk.com/topic-233840277_55344402

https://vk.com/topic-233840280_55144496

https://vk.com/topic-233840278_54550821

https://vk.com/topic-233840271_55622023

https://vk.com/topic-233840279_54771294

https://vk.com/topic-233840274_54059652

https://vk.com/topic-233840273_54357686

https://vk.com/topic-233840272_54731701

https://vk.com/topic-233840276_54431250

https://vk.com/topic-233840275_53945016

https://vk.com/topic-233840277_55344401

https://vk.com/topic-233840280_55144495

https://vk.com/topic-233840278_54550820

https://vk.com/topic-233840271_55622022

https://vk.com/topic-233840279_54771292

https://vk.com/topic-233840274_54059651

https://vk.com/topic-233840273_54357685

https://vk.com/topic-233840272_54731700

https://vk.com/topic-233840276_54431249

https://vk.com/topic-233840275_53945015

https://vk.com/topic-233840277_55344400

https://vk.com/topic-233840280_55144494

https://vk.com/topic-233840278_54550819

https://vk.com/topic-233840271_55622021

https://vk.com/topic-233840279_54771291

https://vk.com/topic-233840274_54059650

https://vk.com/topic-233840273_54357683

https://vk.com/topic-233840272_54731699

https://vk.com/topic-233840276_54431248

https://vk.com/topic-233840275_53945014

https://vk.com/topic-233840277_55344397

https://vk.com/topic-233840280_55144493

https://vk.com/topic-233840278_54550818

https://vk.com/topic-233840271_55622020

https://vk.com/topic-233840279_54771290

https://vk.com/topic-233840274_54059649

https://vk.com/topic-233840273_54357682

https://vk.com/topic-233840272_54731698

https://vk.com/topic-233840276_54431247

https://vk.com/topic-233840275_53945013

https://vk.com/topic-233840277_55344396

https://vk.com/topic-233840280_55144492

https://vk.com/topic-233840278_54550817

https://vk.com/topic-233840271_55622019

https://vk.com/topic-233840279_54771288

https://vk.com/topic-233840274_54059648

https://vk.com/topic-233840273_54357681

https://vk.com/topic-233840272_54731697

https://vk.com/topic-233840276_54431246

https://vk.com/topic-233840275_53945012

https://vk.com/topic-233840277_55344395

https://vk.com/topic-233840280_55144490

https://vk.com/topic-233840278_54550816

https://vk.com/topic-233840271_55622018

https://vk.com/topic-233840279_54771287

https://vk.com/topic-233840274_54059647

https://vk.com/topic-233840273_54357679

https://vk.com/topic-233840272_54731696

https://vk.com/topic-233840276_54431245

https://vk.com/topic-233840275_53945011

https://vk.com/topic-233840277_55344394

https://vk.com/topic-233840280_55144488

https://vk.com/topic-233840278_54550814

https://vk.com/topic-233840271_55622016

https://vk.com/topic-233840279_54771286

https://vk.com/topic-233840274_54059646

https://vk.com/topic-233840273_54357677

https://vk.com/topic-233840272_54731695

https://vk.com/topic-233840276_54431244

https://vk.com/topic-233840275_53945010

https://vk.com/topic-233840277_55344391

https://vk.com/topic-233840280_55144487

https://vk.com/topic-233840278_54550812

https://vk.com/topic-233840271_55622014

https://vk.com/topic-233840279_54771284

https://vk.com/topic-233840274_54059645

https://vk.com/topic-233840273_54357675

https://vk.com/topic-233840272_54731694

https://vk.com/topic-233840276_54431243

https://vk.com/topic-233840275_53945009

https://vk.com/topic-233840277_55344390

https://vk.com/topic-233840280_55144486

https://vk.com/topic-233840278_54550811

https://vk.com/topic-233840271_55622012

https://vk.com/topic-233840279_54771283

https://vk.com/topic-233840274_54059644

https://vk.com/topic-233840273_54357674

https://vk.com/topic-233840272_54731693

https://vk.com/topic-233840276_54431241

https://vk.com/topic-233840275_53945008

https://vk.com/topic-233840277_55344389

https://vk.com/topic-233840280_55144485

https://vk.com/topic-233840278_54550810

https://vk.com/topic-233840271_55622011

https://vk.com/topic-233840279_54771282

https://vk.com/topic-233840274_54059643

https://vk.com/topic-233840273_54357672

https://vk.com/topic-233840272_54731692

https://vk.com/topic-233840276_54431239

https://vk.com/topic-233840275_53945007

https://vk.com/topic-233840277_55344387

https://vk.com/topic-233840280_55144484

https://vk.com/topic-233840278_54550808

https://vk.com/topic-233840271_55622008

https://vk.com/topic-233840279_54771281

https://vk.com/topic-233840274_54059642

https://vk.com/topic-233840273_54357670

https://vk.com/topic-233840272_54731691

https://vk.com/topic-233840276_54431237

https://vk.com/topic-233840275_53945006

https://vk.com/topic-233840277_55344385

https://vk.com/topic-233840280_55144483

https://vk.com/topic-233840278_54550805

https://vk.com/topic-233840271_55622007

https://vk.com/topic-233840279_54771280

https://vk.com/topic-233840274_54059641

https://vk.com/topic-233840273_54357669

https://vk.com/topic-233840272_54731690

https://vk.com/topic-233840276_54431235

https://vk.com/topic-233840275_53945003

https://vk.com/topic-233840277_55344383

https://vk.com/topic-233840280_55144482

https://vk.com/topic-233840278_54550802

https://vk.com/topic-233840271_55622005

https://vk.com/topic-233840279_54771279

https://vk.com/topic-233840274_54059638

https://vk.com/topic-233840273_54357665

https://vk.com/topic-233840272_54731689

https://vk.com/topic-233840276_54431234

https://vk.com/topic-233840277_55344381

https://vk.com/topic-233840275_53945002

https://vk.com/topic-233840280_55144481

https://vk.com/topic-233840278_54550800

https://vk.com/topic-233840271_55622003

https://vk.com/topic-233840279_54771277

https://vk.com/topic-233840274_54059637

https://vk.com/topic-233840273_54357664

https://vk.com/topic-233840272_54731688

https://vk.com/topic-233840276_54431232

https://vk.com/topic-233840277_55344379

https://vk.com/topic-233840275_53945001

https://vk.com/topic-233840280_55144479

https://vk.com/topic-233840278_54550799

https://vk.com/topic-233840271_55621997

https://vk.com/topic-233840279_54771275

https://vk.com/topic-233840274_54059636

https://vk.com/topic-233840273_54357663

https://vk.com/topic-233840272_54731687

https://vk.com/topic-233840276_54431231

https://vk.com/topic-233840277_55344378

https://vk.com/topic-233840275_53945000

https://vk.com/topic-233840280_55144478

https://vk.com/topic-233840278_54550797

https://vk.com/topic-233840271_55621996

https://vk.com/topic-233840279_54771274

https://vk.com/topic-233840274_54059635

https://vk.com/topic-233840273_54357662

https://vk.com/topic-233840272_54731685

https://vk.com/topic-233840276_54431230

https://vk.com/topic-233840277_55344377

https://vk.com/topic-233840275_53944999

https://vk.com/topic-233840280_55144476

https://vk.com/topic-233840278_54550796

https://vk.com/topic-233840271_55621995

https://vk.com/topic-233840279_54771273

https://vk.com/topic-233840274_54059634

https://vk.com/topic-233840273_54357660

https://vk.com/topic-233840272_54731684

https://vk.com/topic-233840276_54431228

https://vk.com/topic-233840275_53944998

https://vk.com/topic-233840277_55344373

https://vk.com/topic-233840280_55144475

https://vk.com/topic-233840278_54550795

https://vk.com/topic-233840271_55621994

https://vk.com/topic-233840279_54771271

https://vk.com/topic-233840274_54059633

https://vk.com/topic-233840273_54357658

https://vk.com/topic-233840272_54731683

https://vk.com/topic-233840276_54431227

https://vk.com/topic-233840275_53944997

https://vk.com/topic-233840277_55344372

https://vk.com/topic-233840280_55144474

https://vk.com/topic-233840278_54550794

https://vk.com/topic-233840271_55621993

https://vk.com/topic-233840279_54771269

https://vk.com/topic-233840274_54059632

https://vk.com/topic-233840273_54357657

https://vk.com/topic-233840272_54731682

https://vk.com/topic-233840276_54431226

https://vk.com/topic-233840275_53944996

https://vk.com/topic-233840277_55344369

https://vk.com/topic-233840280_55144473

https://vk.com/topic-233840278_54550792

https://vk.com/topic-233840271_55621992

https://vk.com/topic-233840279_54771268

https://vk.com/topic-233840274_54059631

https://vk.com/topic-233840273_54357655

https://vk.com/topic-233840272_54731681

https://vk.com/topic-233840276_54431223

https://vk.com/topic-233840277_55344368

https://vk.com/topic-233840275_53944995

https://vk.com/topic-233840280_55144472

https://vk.com/topic-233840278_54550791

https://vk.com/topic-233840271_55621991

https://vk.com/topic-233840279_54771267

https://vk.com/topic-233840274_54059630

https://vk.com/topic-233840273_54357654

https://vk.com/topic-233840272_54731680

https://vk.com/topic-233840276_54431222

https://vk.com/topic-233840277_55344367

https://vk.com/topic-233840275_53944994

https://vk.com/topic-233840280_55144471

https://vk.com/topic-233840278_54550790

https://vk.com/topic-233840271_55621990

https://vk.com/topic-233840279_54771266

https://vk.com/topic-233840274_54059629

https://vk.com/topic-233840273_54357653

https://vk.com/topic-233840272_54731679

https://vk.com/topic-233840276_54431221

https://vk.com/topic-233840275_53944993

https://vk.com/topic-233840277_55344365

https://vk.com/topic-233840280_55144470

https://vk.com/topic-233840278_54550789

https://vk.com/topic-233840271_55621988

https://vk.com/topic-233840279_54771265

https://vk.com/topic-233840274_54059628

https://vk.com/topic-233840273_54357651

https://vk.com/topic-233840272_54731678

https://vk.com/topic-233840276_54431220

https://vk.com/topic-233840275_53944992

https://vk.com/topic-233840277_55344363

https://vk.com/topic-233840280_55144469

https://vk.com/topic-233840278_54550788

https://vk.com/topic-233840271_55621987

https://vk.com/topic-233840279_54771263

https://vk.com/topic-233840274_54059627

https://vk.com/topic-233840273_54357650

https://vk.com/topic-233840272_54731677

https://vk.com/topic-233840276_54431219

https://vk.com/topic-233840275_53944991

https://vk.com/topic-233840277_55344362

https://vk.com/topic-233840280_55144468

https://vk.com/topic-233840278_54550787

https://vk.com/topic-233840271_55621986

https://vk.com/topic-233840279_54771262

https://vk.com/topic-233840274_54059625

https://vk.com/topic-233840273_54357649

https://vk.com/topic-233840272_54731676

https://vk.com/topic-233840276_54431218

https://vk.com/topic-233840275_53944990

https://vk.com/topic-233840277_55344361

https://vk.com/topic-233840280_55144466

https://vk.com/topic-233840278_54550786

https://vk.com/topic-233840271_55621984

https://vk.com/topic-233840279_54771260

https://vk.com/topic-233840274_54059624

https://vk.com/topic-233840273_54357648

https://vk.com/topic-233840276_54431217

https://vk.com/topic-233840272_54731675

https://vk.com/topic-233840275_53944989

https://vk.com/topic-233840277_55344360

https://vk.com/topic-233840280_55144465

https://vk.com/topic-233840278_54550785

https://vk.com/topic-233840271_55621983

https://vk.com/topic-233840279_54771259

https://vk.com/topic-233840274_54059623

https://vk.com/topic-233840273_54357646

https://vk.com/topic-233840276_54431216

https://vk.com/topic-233840272_54731674

https://vk.com/topic-233840277_55344359

https://vk.com/topic-233840275_53944987

https://vk.com/topic-233840280_55144464

https://vk.com/topic-233840278_54550784

https://vk.com/topic-233840271_55621982

https://vk.com/topic-233840279_54771258

https://vk.com/topic-233840274_54059622

https://vk.com/topic-233840273_54357645

https://vk.com/topic-233840276_54431214

https://vk.com/topic-233840272_54731673

https://vk.com/topic-233840277_55344358

https://vk.com/topic-233840275_53944986

https://vk.com/topic-233840280_55144463

https://vk.com/topic-233840278_54550783

https://vk.com/topic-233840271_55621981

https://vk.com/topic-233840279_54771257

https://vk.com/topic-233840274_54059621

https://vk.com/topic-233840273_54357644

https://vk.com/topic-233840276_54431213

https://vk.com/topic-233840272_54731672

https://vk.com/topic-233840277_55344357

https://vk.com/topic-233840275_53944985

https://vk.com/topic-233840278_54550781

https://vk.com/topic-233840280_55144462

https://vk.com/topic-233840271_55621980

https://vk.com/topic-233840279_54771255

https://vk.com/topic-233840274_54059620

https://vk.com/topic-233840273_54357643

https://vk.com/topic-233840276_54431212

https://vk.com/topic-233840272_54731671

https://vk.com/topic-233840277_55344356

https://vk.com/topic-233840275_53944984

https://vk.com/topic-233840278_54550780

https://vk.com/topic-233840280_55144461

https://vk.com/topic-233840271_55621979

https://vk.com/topic-233840279_54771253

https://vk.com/topic-233840274_54059619

https://vk.com/topic-233840273_54357642

https://vk.com/topic-233840272_54731670

https://vk.com/topic-233840276_54431211

https://vk.com/topic-233840277_55344355

https://vk.com/topic-233840275_53944983

https://vk.com/topic-233840278_54550778

https://vk.com/topic-233840280_55144460

https://vk.com/topic-233840271_55621978

https://vk.com/topic-233840279_54771251

https://vk.com/topic-233840274_54059618

https://vk.com/topic-233840273_54357640

https://vk.com/topic-233840276_54431210

https://vk.com/topic-233840272_54731669

https://vk.com/topic-233840277_55344352

https://vk.com/topic-233840275_53944982

https://vk.com/topic-233840278_54550777

https://vk.com/topic-233840280_55144459

https://vk.com/topic-233840271_55621976

https://vk.com/topic-233840279_54771248

https://vk.com/topic-233840274_54059617

https://vk.com/topic-233840273_54357638

https://vk.com/topic-233840276_54431209

https://vk.com/topic-233840272_54731668

https://vk.com/topic-233840277_55344351

https://vk.com/topic-233840275_53944981

https://vk.com/topic-233840278_54550776

https://vk.com/topic-233840280_55144458

https://vk.com/topic-233840271_55621975

https://vk.com/topic-233840279_54771247

https://vk.com/topic-233840274_54059616

https://vk.com/topic-233840273_54357636

https://vk.com/topic-233840276_54431207

https://vk.com/topic-233840272_54731667

https://vk.com/topic-233840277_55344348

https://vk.com/topic-233840275_53944980

https://vk.com/topic-233840278_54550775

https://vk.com/topic-233840280_55144457

https://vk.com/topic-233840271_55621974

https://vk.com/topic-233840279_54771246

https://vk.com/topic-233840274_54059614

https://vk.com/topic-233840273_54357635

https://vk.com/topic-233840276_54431206

https://vk.com/topic-233840272_54731666

https://vk.com/topic-233840277_55344347

https://vk.com/topic-233840275_53944979

https://vk.com/topic-233840278_54550774

https://vk.com/topic-233840280_55144456

https://vk.com/topic-233840271_55621972

https://vk.com/topic-233840279_54771244

https://vk.com/topic-233840274_54059613

https://vk.com/topic-233840273_54357633

https://vk.com/topic-233840276_54431205

https://vk.com/topic-233840272_54731665

https://vk.com/topic-233840277_55344345

https://vk.com/topic-233840275_53944978

https://vk.com/topic-233840278_54550773

https://vk.com/topic-233840280_55144455

https://vk.com/topic-233840271_55621971

https://vk.com/topic-233840279_54771243

https://vk.com/topic-233840274_54059612

https://vk.com/topic-233840273_54357631

https://vk.com/topic-233840276_54431204

https://vk.com/topic-233840272_54731664

https://vk.com/topic-233840277_55344344

https://vk.com/topic-233840275_53944977

https://vk.com/topic-233840278_54550771

https://vk.com/topic-233840280_55144454

https://vk.com/topic-233840271_55621970

https://vk.com/topic-233840279_54771242

https://vk.com/topic-233840274_54059611

https://vk.com/topic-233840273_54357630

https://vk.com/topic-233840276_54431203

https://vk.com/topic-233840272_54731663

https://vk.com/topic-233840277_55344343

https://vk.com/topic-233840275_53944976

https://vk.com/topic-233840278_54550770

https://vk.com/topic-233840280_55144453

https://vk.com/topic-233840271_55621968

https://vk.com/topic-233840279_54771240

https://vk.com/topic-233840274_54059610

https://vk.com/topic-233840273_54357629

https://vk.com/topic-233840276_54431202

https://vk.com/topic-233840272_54731662

https://vk.com/topic-233840277_55344341

https://vk.com/topic-233840275_53944975

https://vk.com/topic-233840278_54550769

https://vk.com/topic-233840280_55144452

https://vk.com/topic-233840271_55621967

https://vk.com/topic-233840279_54771238

https://vk.com/topic-233840274_54059609

https://vk.com/topic-233840273_54357627

https://vk.com/topic-233840276_54431201

https://vk.com/topic-233840272_54731661

https://vk.com/topic-233840277_55344340

https://vk.com/topic-233840275_53944974

https://vk.com/topic-233840278_54550768

https://vk.com/topic-233840280_55144451

https://vk.com/topic-233840271_55621965

https://vk.com/topic-233840279_54771237

https://vk.com/topic-233840274_54059608

https://vk.com/topic-233840273_54357626

https://vk.com/topic-233840276_54431200

https://vk.com/topic-233840272_54731660

https://vk.com/topic-233840277_55344339

https://vk.com/topic-233840275_53944973

https://vk.com/topic-233840278_54550767

https://vk.com/topic-233840280_55144450

https://vk.com/topic-233840271_55621963

https://vk.com/topic-233840279_54771236

https://vk.com/topic-233840274_54059607

https://vk.com/topic-233840273_54357625

https://vk.com/topic-233840276_54431199

https://vk.com/topic-233840272_54731659

https://vk.com/topic-233840277_55344338

https://vk.com/topic-233840275_53944972

https://vk.com/topic-233840278_54550766

https://vk.com/topic-233840280_55144449

https://vk.com/topic-233840271_55621961

https://vk.com/topic-233840279_54771235

https://vk.com/topic-233840274_54059606

https://vk.com/topic-233840273_54357623

https://vk.com/topic-233840276_54431198

https://vk.com/topic-233840272_54731658

https://vk.com/topic-233840277_55344337

https://vk.com/topic-233840275_53944971

https://vk.com/topic-233840278_54550765

https://vk.com/topic-233840280_55144448

https://vk.com/topic-233840271_55621960

https://vk.com/topic-233840279_54771234

https://vk.com/topic-233840274_54059605

https://vk.com/topic-233840273_54357621

https://vk.com/topic-233840276_54431197

https://vk.com/topic-233840272_54731657

https://vk.com/topic-233840277_55344335

https://vk.com/topic-233840275_53944970

https://vk.com/topic-233840278_54550764

https://vk.com/topic-233840280_55144447

https://vk.com/topic-233840271_55621959

https://vk.com/topic-233840279_54771232

https://vk.com/topic-233840274_54059604

https://vk.com/topic-233840273_54357620

https://vk.com/topic-233840276_54431196

https://vk.com/topic-233840272_54731656

https://vk.com/topic-233840277_55344334

https://vk.com/topic-233840275_53944969

https://vk.com/topic-233840278_54550763

https://vk.com/topic-233840280_55144446

https://vk.com/topic-233840271_55621958

https://vk.com/topic-233840274_54059603

https://vk.com/topic-233840279_54771230

https://vk.com/topic-233840273_54357619

https://vk.com/topic-233840276_54431195

https://vk.com/topic-233840272_54731655

https://vk.com/topic-233840277_55344332

https://vk.com/topic-233840275_53944968

https://vk.com/topic-233840278_54550762

https://vk.com/topic-233840280_55144445

https://vk.com/topic-233840271_55621957

https://vk.com/topic-233840274_54059602

https://vk.com/topic-233840279_54771229

https://vk.com/topic-233840273_54357617

https://vk.com/topic-233840276_54431194

https://vk.com/topic-233840272_54731652

https://vk.com/topic-233840277_55344330

https://vk.com/topic-233840275_53944965

https://vk.com/topic-233840278_54550761

https://vk.com/topic-233840280_55144444

https://vk.com/topic-233840271_55621952

https://vk.com/topic-233840274_54059601

https://vk.com/topic-233840279_54771228

https://vk.com/topic-233840273_54357616

https://vk.com/topic-233840276_54431193

https://vk.com/topic-233840272_54731651

https://vk.com/topic-233840277_55344329

https://vk.com/topic-233840275_53944964

https://vk.com/topic-233840280_55144443

https://vk.com/topic-233840278_54550760

https://vk.com/topic-233840271_55621951

https://vk.com/topic-233840274_54059600

https://vk.com/topic-233840279_54771227

https://vk.com/topic-233840273_54357614

https://vk.com/topic-233840276_54431192

https://vk.com/topic-233840272_54731650

https://vk.com/topic-233840277_55344327

https://vk.com/topic-233840280_55144440

https://vk.com/topic-233840278_54550759

https://vk.com/topic-233840271_55621949

https://vk.com/topic-233840279_54771225

https://vk.com/topic-233840274_54059599

https://vk.com/topic-233840273_54357611

https://vk.com/topic-233805839_54771076

https://vk.com/topic-233839834_54059563

https://vk.com/topic-233839836_54431111

https://vk.com/topic-233839835_53944945

https://vk.com/topic-233839837_55344183

https://vk.com/topic-233805839_54771075

https://vk.com/topic-233839834_54059562

https://vk.com/topic-233839835_53944944

https://vk.com/topic-233839836_54431110

https://vk.com/topic-233839837_55344182

https://vk.com/topic-233805839_54771074

https://vk.com/topic-233839835_53944943

https://vk.com/topic-233839834_54059561

https://vk.com/topic-233839836_54431109

https://vk.com/topic-233839837_55344181

https://vk.com/topic-233805839_54771072

https://vk.com/topic-233839835_53944942

https://vk.com/topic-233839834_54059560

https://vk.com/topic-233839836_54431108

https://vk.com/topic-233839837_55344177

https://vk.com/topic-233805839_54771071

https://vk.com/topic-233839835_53944941

https://vk.com/topic-233839834_54059558

https://vk.com/topic-233839836_54431107

https://vk.com/topic-233839837_55344176

https://vk.com/topic-233805839_54771070

https://vk.com/topic-233839835_53944939

https://vk.com/topic-233839834_54059557

https://vk.com/topic-233839836_54431105

https://vk.com/topic-233839837_55344175

https://vk.com/topic-233805839_54771067

https://vk.com/topic-233839835_53944938

https://vk.com/topic-233839834_54059556

https://vk.com/topic-233839836_54431103

https://vk.com/topic-233839837_55344174

https://vk.com/topic-233805839_54771066

https://vk.com/topic-233839835_53944937

https://vk.com/topic-233839834_54059555

https://vk.com/topic-233839836_54431102

https://vk.com/topic-233839837_55344173

https://vk.com/topic-233805839_54771065

https://vk.com/topic-233839835_53944936

https://vk.com/topic-233839836_54431101

https://vk.com/topic-233839834_54059554

https://vk.com/topic-233839837_55344171

https://vk.com/topic-233805839_54771063

https://vk.com/topic-233839835_53944935

https://vk.com/topic-233839836_54431100

https://vk.com/topic-233839834_54059553

https://vk.com/topic-233839837_55344170

https://vk.com/topic-233805839_54771062

https://vk.com/topic-233839836_54431098

https://vk.com/topic-233839835_53944934

https://vk.com/topic-233839834_54059552

https://vk.com/topic-233839837_55344169

https://vk.com/topic-233805839_54771060

https://vk.com/topic-233839836_54431097

https://vk.com/topic-233839835_53944933

https://vk.com/topic-233839834_54059551

https://vk.com/topic-233839837_55344168

https://vk.com/topic-233805839_54771059

https://vk.com/topic-233839836_54431096

https://vk.com/topic-233839834_54059549

https://vk.com/topic-233839835_53944932

https://vk.com/topic-233839837_55344167

https://vk.com/topic-233805839_54771058

https://vk.com/topic-233839836_54431095

https://vk.com/topic-233839834_54059548

https://vk.com/topic-233839835_53944931

https://vk.com/topic-233839837_55344166

https://vk.com/topic-233805839_54771055

https://vk.com/topic-233839836_54431094

https://vk.com/topic-233839834_54059547

https://vk.com/topic-233839835_53944930

https://vk.com/topic-233839837_55344165

https://vk.com/topic-233805839_54771054

https://vk.com/topic-233839836_54431093

https://vk.com/topic-233839835_53944929

https://vk.com/topic-233839834_54059545

https://vk.com/topic-233839837_55344163

https://vk.com/topic-233805839_54771053

https://vk.com/topic-233839836_54431090

https://vk.com/topic-233839835_53944928

https://vk.com/topic-233839834_54059544

https://vk.com/topic-233839837_55344161

https://vk.com/topic-233805839_54771052

https://vk.com/topic-233839836_54431089

https://vk.com/topic-233839834_54059543

https://vk.com/topic-233839835_53944927

https://vk.com/topic-233839837_55344160

https://vk.com/topic-233805839_54771050

https://vk.com/topic-233839836_54431088

https://vk.com/topic-233839837_55344158

https://vk.com/topic-233839835_53944926

https://vk.com/topic-233839834_54059542

https://vk.com/topic-233805839_54771048

https://vk.com/topic-233839836_54431087

https://vk.com/topic-233839835_53944925

https://vk.com/topic-233839837_55344157

https://vk.com/topic-233839834_54059541

https://vk.com/topic-233805839_54771047

https://vk.com/topic-233839836_54431084

https://vk.com/topic-233839835_53944924

https://vk.com/topic-233839837_55344155

https://vk.com/topic-233839834_54059540

https://vk.com/topic-233805839_54771046

https://vk.com/topic-233839836_54431083

https://vk.com/topic-233839835_53944923

https://vk.com/topic-233839837_55344154

https://vk.com/topic-233839834_54059539

https://vk.com/topic-233805839_54771044

https://vk.com/topic-233839836_54431082

https://vk.com/topic-233839837_55344151

https://vk.com/topic-233839835_53944922

https://vk.com/topic-233839834_54059538

https://vk.com/topic-233805839_54771043

https://vk.com/topic-233839836_54431081

https://vk.com/topic-233839837_55344150

https://vk.com/topic-233839835_53944921

https://vk.com/topic-233839834_54059537

https://vk.com/topic-233805839_54771042

https://vk.com/topic-233839837_55344148

https://vk.com/topic-233839836_54431079

https://vk.com/topic-233839835_53944920

https://vk.com/topic-233839834_54059536

https://vk.com/topic-233805839_54771040

https://vk.com/topic-233839837_55344147

https://vk.com/topic-233839836_54431078

https://vk.com/topic-233839835_53944919

https://vk.com/topic-233839834_54059535

https://vk.com/topic-233805839_54771039

https://vk.com/topic-233839837_55344146

https://vk.com/topic-233839836_54431077

https://vk.com/topic-233839835_53944917

https://vk.com/topic-233839834_54059534

https://vk.com/topic-233805839_54771037

https://vk.com/topic-233839837_55344142

https://vk.com/topic-233839836_54431076

https://vk.com/topic-233839835_53944916

https://vk.com/topic-233839834_54059533

https://vk.com/topic-233805839_54771036

https://vk.com/topic-233839837_55344141

https://vk.com/topic-233839836_54431075

https://vk.com/topic-233839835_53944915

https://vk.com/topic-233839834_54059531

https://vk.com/topic-233805839_54771034

https://vk.com/topic-233839837_55344140

https://vk.com/topic-233839836_54431074

https://vk.com/topic-233839835_53944914

https://vk.com/topic-233839834_54059530

https://vk.com/topic-233805839_54771033

https://vk.com/topic-233839837_55344138

https://vk.com/topic-233839836_54431072

https://vk.com/topic-233839835_53944913

https://vk.com/topic-233839834_54059529

https://vk.com/topic-233805839_54771032

https://vk.com/topic-233839837_55344137

https://vk.com/topic-233839836_54431071

https://vk.com/topic-233839835_53944912

https://vk.com/topic-233839834_54059528

https://vk.com/topic-233805839_54771030

https://vk.com/topic-233839837_55344135

https://vk.com/topic-233839836_54431070

https://vk.com/topic-233839834_54059527

https://vk.com/topic-233839835_53944911

https://vk.com/topic-233805839_54771029

https://vk.com/topic-233839837_55344134

https://vk.com/topic-233839836_54431069

https://vk.com/topic-233839834_54059526

https://vk.com/topic-233839835_53944910

https://vk.com/topic-233805839_54771028

https://vk.com/topic-233839837_55344132

https://vk.com/topic-233839836_54431068

https://vk.com/topic-233839835_53944909

https://vk.com/topic-233839834_54059525

https://vk.com/topic-233805839_54771027

https://vk.com/topic-233839837_55344131

https://vk.com/topic-233839836_54431067

https://vk.com/topic-233839835_53944907

https://vk.com/topic-233839834_54059524

https://vk.com/topic-233805839_54771024

https://vk.com/topic-233839837_55344130

https://vk.com/topic-233839836_54431066

https://vk.com/topic-233839835_53944906

https://vk.com/topic-233839834_54059523

https://vk.com/topic-233805839_54771023

https://vk.com/topic-233839837_55344128

https://vk.com/topic-233839835_53944905

https://vk.com/topic-233839836_54431065

https://vk.com/topic-233839834_54059522

https://vk.com/topic-233805839_54771021

https://vk.com/topic-233839837_55344127

https://vk.com/topic-233839835_53944904

https://vk.com/topic-233839836_54431064

https://vk.com/topic-233839834_54059521

https://vk.com/topic-233805839_54771018

https://vk.com/topic-233839837_55344125

https://vk.com/topic-233839835_53944903

https://vk.com/topic-233839836_54431063

https://vk.com/topic-233839834_54059520

https://vk.com/topic-233805839_54771017

https://vk.com/topic-233839837_55344124

https://vk.com/topic-233839835_53944902

https://vk.com/topic-233839836_54431062

https://vk.com/topic-233839834_54059519

https://vk.com/topic-233805839_54771016

https://vk.com/topic-233839837_55344123

https://vk.com/topic-233839835_53944901

https://vk.com/topic-233839836_54431060

https://vk.com/topic-233839834_54059518

https://vk.com/topic-233805839_54771014

https://vk.com/topic-233839837_55344121

https://vk.com/topic-233839836_54431059

https://vk.com/topic-233839835_53944900

https://vk.com/topic-233839834_54059517

https://vk.com/topic-233805839_54771013

https://vk.com/topic-233839837_55344119

https://vk.com/topic-233839836_54431058

https://vk.com/topic-233839835_53944899

https://vk.com/topic-233839834_54059516

https://vk.com/topic-233805839_54771012

https://vk.com/topic-233839837_55344118

https://vk.com/topic-233839836_54431057

https://vk.com/topic-233839835_53944898

https://vk.com/topic-233839834_54059515

https://vk.com/topic-233805839_54771011

https://vk.com/topic-233839837_55344115

https://vk.com/topic-233839836_54431055

https://vk.com/topic-233839835_53944897

https://vk.com/topic-233839834_54059514

https://vk.com/topic-233805839_54771009

https://vk.com/topic-233839837_55344113

https://vk.com/topic-233839835_53944896

https://vk.com/topic-233839836_54431054

https://vk.com/topic-233839834_54059512

https://vk.com/topic-233805839_54771008

https://vk.com/topic-233839837_55344112

https://vk.com/topic-233839835_53944895

https://vk.com/topic-233839836_54431053

https://vk.com/topic-233839834_54059511

https://vk.com/topic-233805839_54771007

https://vk.com/topic-233839837_55344109

https://vk.com/topic-233839835_53944894

https://vk.com/topic-233839836_54431051

https://vk.com/topic-233839834_54059510

https://vk.com/topic-233805839_54771005

https://vk.com/topic-233839837_55344107

https://vk.com/topic-233839836_54431050

https://vk.com/topic-233839835_53944893

https://vk.com/topic-233839834_54059508

https://vk.com/topic-233839746_54431046

https://vk.com/topic-233839744_54059505

https://vk.com/topic-233839743_54357437

https://vk.com/topic-233839742_54731621

https://vk.com/topic-233839747_55344104

https://vk.com/topic-233839745_53944891

https://vk.com/topic-233839740_55144321

https://vk.com/topic-233839739_54770999

https://vk.com/topic-233839737_55344103

https://vk.com/topic-233839738_54550696

https://vk.com/topic-233839746_54431045

https://vk.com/topic-233839744_54059504

https://vk.com/topic-233839743_54357435

https://vk.com/topic-233839742_54731620

https://vk.com/topic-233839745_53944890

https://vk.com/topic-233839747_55344102

https://vk.com/topic-233839740_55144320

https://vk.com/topic-233839739_54770998

https://vk.com/topic-233839737_55344100

https://vk.com/topic-233839738_54550695

https://vk.com/topic-233839746_54431044

https://vk.com/topic-233839744_54059502

https://vk.com/topic-233839742_54731619

https://vk.com/topic-233839743_54357434

https://vk.com/topic-233839745_53944889

https://vk.com/topic-233839747_55344099

https://vk.com/topic-233839740_55144319

https://vk.com/topic-233839739_54770997

https://vk.com/topic-233839737_55344097

https://vk.com/topic-233839738_54550693

https://vk.com/topic-233839746_54431043

https://vk.com/topic-233839744_54059501

https://vk.com/topic-233839742_54731618

https://vk.com/topic-233839743_54357433

https://vk.com/topic-233839745_53944888

https://vk.com/topic-233839747_55344096

https://vk.com/topic-233839740_55144318

https://vk.com/topic-233839739_54770996

https://vk.com/topic-233839737_55344095

https://vk.com/topic-233839738_54550692

https://vk.com/topic-233839746_54431042

https://vk.com/topic-233839744_54059500

https://vk.com/topic-233839742_54731616

https://vk.com/topic-233839743_54357431

https://vk.com/topic-233839745_53944887

https://vk.com/topic-233839747_55344093

https://vk.com/topic-233839740_55144317

https://vk.com/topic-233839739_54770994

https://vk.com/topic-233839737_55344092

https://vk.com/topic-233839738_54550691

https://vk.com/topic-233839746_54431040

https://vk.com/topic-233839744_54059499

https://vk.com/topic-233839742_54731615

https://vk.com/topic-233839743_54357430

from langgraph.graph import Graph, END

def rl_teach(state):

obs = torch.tensor(state['urgencies'], dtype=torch.float32)

probs, _ = policy(obs)

action = torch.argmax(probs).item() # greedy selectionreturn {"next_lesson": action}

graph = Graph()

graph.add_node("assess_student", lambda state: {"urgencies": np.random.rand(3)})

graph.add_node("plan_lesson", rl_teach)

graph.add_node("deliver_content", lambda state: {"done": True, "taught": state["next_lesson"]})

graph.add_edge("assess_student", "plan_lesson")

graph.add_edge("plan_lesson", "deliver_content")

graph.add_edge("deliver_content", END)

compiled_graph = graph.compile()

result = compiled_graph.invoke({})

print("Lesson Plan:", result) # e.g., {'done': True, 'taught': 2}Graph 构建要点：

rl_teach 是核心决策节点，读取 state 里的 urgencies，用训练好的 policy 推理，选概率最高的 action（部署时用 greedy，训练时要 exploration）

节点用 lambda 定义简单任务，复杂逻辑就定义函数

add_edge 定义数据流向；也可以用 add_conditional_edges 加条件分支（比如"if urgency>0.7, rush course"）

compile()把图结构编译成可执行对象，invoke()触发运行

这样就完成了。

如果我们后续扩展，可以加个 LLM 节点生成 lesson explanation，在真实学生数据上训练 policy，或者接入更复杂的 reward shaping（考虑学生历史表现、知识依赖关系等）。训练后的 Agent 会自动 prioritize 高 urgency topic，不需要手动写规则。

总结 RL 不是什么黑科技，但确实能让 Agent 在不确定环境里真正 intelligent 起来。配合 LangGraph 这类框架，可以把 RL 无缝嵌入实际业务流程。RL 和 LLM 结合（比如 RLHF 做 alignment）能搞出既会优化路线，又能用自然语言解释决策的混合 agent——这才是 agentic AI 该有的样子。