写点什么

多线程 VS 多进程(三)

用户头像
若尘
关注
发布于: 2021 年 05 月 21 日
多线程 VS 多进程(三)

线程替代方案

  • subprocess

  • 完全跳过线程,使用进程

  • 是派生进程的主要替代方案

  • python2.4 后引入

  • multiprocessiong

  • 使用 threading 接口派生,使用子进程

  • 允许为多核或者多 cpu 派生进程,接口跟 threading 非常相似

  • python2.6 后引入

  • concurrent.futures

  • 新的异步执行模块

  • 任务级别的操作

  • python3.2 后引入

多进程

  • 进程间通讯(InterprocessCommunication, IPC)

  • 进程之间无任何共享状态

  • 进程的创建

  • 直接生成 Process 实例对象,案例 19


      import multiprocessing    from time import sleep, ctime        def clock(interval):        while True:            print("The time is %s" % ctime())            sleep(interval)        if __name__ == '__main__':        p = multiprocessing.Process(target=clock, args=(5, ))        p.start()            while True:            print("sleeping......")            sleep(1)
复制代码


      sleeping......    The time is Tue Aug 13 19:47:41 2019    sleeping......    sleeping......    sleeping......    sleeping......    sleeping......    The time is Tue Aug 13 19:47:46 2019    sleeping......    sleeping......    sleeping......    sleeping......    ...- 派生子类,案例20
复制代码


      import multiprocessing    from time import sleep, ctime        class ClockProcess(multiprocessing.Process):        '''        两个函数比较重要        1. init构造函数        2. run        '''        def __init__(self, interval):            super(ClockProcess, self).__init__()            self.interval = interval        def run(self):            while True:                print("The time is %s" % ctime())                sleep(self.interval)        if __name__ == '__main__':        p = ClockProcess(3)        p.start()
复制代码


      The time is Tue Aug 13 19:48:49 2019    The time is Tue Aug 13 19:48:52 2019    The time is Tue Aug 13 19:48:55 2019    The time is Tue Aug 13 19:48:58 2019    The time is Tue Aug 13 19:49:01 2019    The time is Tue Aug 13 19:49:04 2019    ...
复制代码


  • 在 os 中查看 pid,ppid 以及他们的关系

  • 案例 21


      from multiprocessing import Process    import os        def info(title):        print(title)        print("module name:", __name__)        # 得到父亲进程的id        print("parent process:", os.getppid())        # 得到本身进程的id        print("process id:", os.getpid())        def f(name):        info('function f')        print('hello', name)        if __name__ == '__main__':        info('main line')        p = Process(target=f, args=('bob', ))        p.start()        p.join()
复制代码


      main line    module name: __main__    parent process: 11900    process id: 18832    function f    module name: __mp_main__    parent process: 18832    process id: 20868    hello bob
复制代码


  • 生产者消费者模型

  • JoinableQueue

  • 案例 22


      import multiprocessing    from time import ctime        def consumer(input_q):        print("Into consumer:", ctime)        while True:            # 处理项            item = input_q.get()            print('pull', item, 'out of q')            input_q.task_done()  # 发出信号通知任务完成        print("Out of consumer:", ctime()) ## 此句未执行,因为q.join()收集到四个task_done()信号后,主进程启动,未等到print此句完成,程序就结束了        def producer(sequence, output_q):        print("Into procuder:", ctime())        for item in sequence:            output_q.put(item)            print('put', item, 'into q')        print('Out of procuder', ctime())        # 建立进程    if __name__ == '__main__':        q = multiprocessing.JoinableQueue()        # 进行消费者进程        cons_p = multiprocessing.Process(target=consumer, args=(q, ))        cons_p.daemon = True        cons_p.start()            # 生产多个项,sequence代表要发送给消费者的项序列        # 在实践中,这可能是生成器的输出或通过一些其他地方生产出来        sequence = [1, 2, 3, 4]        producer(sequence, q)        # 等待所有项被处理        q.join()
复制代码


      Into procuder: Tue Aug 13 19:50:38 2019    put 1 into q    put 2 into q    put 3 into q    put 4 into q    Out of procuder Tue Aug 13 19:50:38 2019    Into consumer: <built-in function ctime>    pull 1 out of q    pull 2 out of q    pull 3 out of q    pull 4 out of q- 队列中哨兵的使用,案例23
复制代码


      import multiprocessing    from time import ctime        # 设置哨兵问题    def consumer(input_q):        print("Into consumer:", ctime())        while True:            item = input_q.get()            if item is None:                break            print("pull", item, "out of q")        print ("Out of consumer:", ctime()) ## 此句执行完成,再转入主进程            def producer(sequence, output_q):        print ("Into procuder:", ctime())        for item in sequence:            output_q.put(item)            print ("put", item, "into q")        print ("Out of procuder:", ctime())        if __name__ == '__main__':        q = multiprocessing.Queue()        cons_p = multiprocessing.Process(target = consumer, args = (q,))        cons_p.start()            sequence = [1,2,3,4]        producer(sequence, q)            q.put(None)        cons_p.join()
复制代码


      Into procuder: Tue Aug 13 19:51:23 2019    put 1 into q    put 2 into q    put 3 into q    put 4 into q    Out of procuder: Tue Aug 13 19:51:23 2019    Into consumer: Tue Aug 13 19:51:24 2019    pull 1 out of q    pull 2 out of q    pull 3 out of q    pull 4 out of q    Out of consumer: Tue Aug 13 19:51:24 2019- 哨兵的改进,案例24
复制代码


      import multiprocessing    from time import ctime        def consumer(input_q):        print ("Into consumer:", ctime())        while True:            item = input_q.get()            if item is None:                break            print("pull", item, "out of q")        print ("Out of consumer:", ctime())        def producer(sequence, output_q):        for item in sequence:            print ("Into procuder:", ctime())            output_q.put(item)            print ("Out of procuder:", ctime())        if __name__ == '__main__':        q = multiprocessing.Queue()        cons_p1 = multiprocessing.Process (target = consumer, args = (q,))        cons_p1.start()            cons_p2 = multiprocessing.Process (target = consumer, args = (q,))        cons_p2.start()            sequence = [1,2,3,4]        producer(sequence, q)            q.put(None)        q.put(None)            cons_p1.join()        cons_p2.join()
复制代码


      Into procuder: Tue Aug 13 19:52:08 2019    Out of procuder: Tue Aug 13 19:52:08 2019    Into procuder: Tue Aug 13 19:52:08 2019    Out of procuder: Tue Aug 13 19:52:08 2019    Into procuder: Tue Aug 13 19:52:08 2019    Out of procuder: Tue Aug 13 19:52:08 2019    Into procuder: Tue Aug 13 19:52:08 2019    Out of procuder: Tue Aug 13 19:52:08 2019    Into consumer: Tue Aug 13 19:52:08 2019    pull 1 out of q    pull 2 out of q    pull 3 out of q    pull 4 out of q    Out of consumer: Tue Aug 13 19:52:08 2019    Into consumer: Tue Aug 13 19:52:08 2019    Out of consumer: Tue Aug 13 19:52:08 2019
复制代码


发布于: 2021 年 05 月 21 日阅读数: 11
用户头像

若尘

关注

还未添加个人签名 2021.01.11 加入

还未添加个人简介

评论

发布
暂无评论
多线程 VS 多进程(三)