Python自动化脚本实现

MySQL导出csv并发送邮件定时执行

参考资料:

mysql导出csv文件excel打开后数字用科学计数法显示且低位变0的解决方法

Include headers when using SELECT INTO OUTFILE?

Connecting to MySQL Using Connector/Python

Installing Connector/Python on Linux Using the MySQL Yum Repository

Tutorial: How to send an email with Python

SMTP发送邮件

How do I send an email with a .csv attachment using Python

背景和目的

用户需要指定时间制定条件查询数据库并导出表格。这个需求涉及到:

  • MySQL导出CSV
  • 发送邮件
  • 定时执行

实现步骤

MySQL导出CSV

导出语句示例:

select "姓名", "性别", "出生日期", "手机", "证件号码", "现居住城市", "最高学位", "报考所用学历", "毕业学校", "专业", "学历证书编号", "学位证书编号", "公司名称", "目前职务", "审核状态" union all select user_name as "姓名", case user_sex when 1 then "女" when 0 then "男" end as "性别", birth as "出生日期", mobile_phone as "手机", id_num as "证件号码", residence_city as "现居住城市", case highest_degree when 0 then "本科" when 1 then "硕士" when 2 then "博士" end as "最高学位", case exam_edu when 0 then "本科" when 1 then "硕士" when 2 then "博士" when 3 then "大专" end as "报考所用学历", graduate as "毕业学校", profess as "专业", edu_num as "学历证书编号", degree_num as "学位证书编号", company_name as "公司名称", curr_job as "目前职位", audit_status from application where audit_status in ('1', '3')  into outfile '/tmp/tsinghua1008.csv' CHARACTER SET gb2312 FIELDS TERMINATED BY ',' ENCLOSED BY '"' optionally enclosed by '\t' LINES TERMINATED BY '\r\n';

需要注意的几个点是:

  • 导出包含表头

    在查询语句前面加上

    select "姓名", ... union all ...
    
  • 根据filed的value显示不同的值,比如性别

    case user_sex when 1 then "女" when 0 then "男" end
    
  • 导出表格Microsoft excel打开乱码

    Microsoft excel默认的编码是gb2312,所以加上CHARACTER SET gb2312

  • 防守打法的发送到发送到

    FIELDS TERMINATED BY ',' ENCLOSED BY '"' optionally enclosed by '\t' LINES TERMINATED BY '\r\n'
    
连接数据库

我们采用mysql.connector

CentOS安装:

$ yum update mysql-community-release
$ yum install mysql-connector-python

连接并执行MySQL语句示例:

import mysql.connector
from mysql.connector import errorcode

config = {
  'user': 'root',
  'password': '******',
  'host': '127.0.0.1',
  'database': 'tsinghua',
  'raise_on_warnings': True,
}

try:
    cnx = mysql.connector.connect(**config)
except mysql.connector.Error as err:
    if err.errno == errorcode.ER_ACCESS_DENIED_ERROR:
        print("Something is wrong with your user name or password")
    elif err.errno == errorcode.ER_BAD_DB_ERROR:
        print("Database does not exist")
    else:
        print(err)

cursor = cnx.cursor()

cursor.execute(sqlStr)
cnx.close()
发送有附件的邮件
import smtplib
import mimetypes
from email.MIMEMultipart import MIMEMultipart
from email.mime.audio import MIMEAudio
from email.mime.base import MIMEBase
from email.mime.image import MIMEImage
from email.mime.text import MIMEText
from email import encoders
from email.utils import parseaddr, formataddr
from email.header import Header

def sendEmail(fromaddr, senderName, toaddr, subject, body, filename,
              fileToSend):
    """
    fromaddr 发件人邮箱
    senderName 发件人姓名
    toaddr 收件人邮箱,多个邮箱用逗号分隔,格式为
            '"yangle@kaoshixing.com","liyang0007@mail.tsinghua.edu.cn"'
    subject 邮件标题
    body 邮件正文
    filename 附件名称
    fileToSend 附件路径
    """
    # 格式化地址函数
    def _format_addr(s):
        name, addr = parseaddr(s)
        return formataddr((Header(name, 'utf-8').encode(),
                           addr.encode('utf-8') if isinstance(addr, unicode)
                           else addr))

    # 邮件内容
    msg = MIMEMultipart()
    # 收件人会看到发件人的姓名,而不只是邮件地址
    msg['From'] = _format_addr(u'%s <%s>' % (senderName, fromaddr))
    msg['To'] = _format_addr(u'<%s>' % toaddr)
    # 邮件标题
    msg['Subject'] = Header(subject, 'utf-8').encode()
    # 邮件正文
    msg.attach(MIMEText(body.encode("utf-8") if isinstance(body, unicode) else
                        body, 'plain'))

    # 处理附件,处理不同的格式
    ctype, encoding = mimetypes.guess_type(fileToSend)
    if ctype is None or encoding is not None:
        ctype = "application/octet-stream"
    maintype, subtype = ctype.split("/", 1)
    if maintype == "text":
        fp = open(fileToSend)
        attachment = MIMEText(fp.read(), _subtype=subtype)
        fp.close()
    elif maintype == "image":
        fp = open(fileToSend, "rb")
        attachment = MIMEImage(fp.read(), _subtype=subtype)
        fp.close()
    elif maintype == "audio":
        fp = open(fileToSend, "rb")
        attachment = MIMEAudio(fp.read(), _subtype=subtype)
        fp.close()
    else:
        fp = open(fileToSend, "rb")
        attachment = MIMEBase(maintype, subtype)
        attachment.set_payload(fp.read())
        fp.close()
        encoders.encode_base64(attachment)
    attachment.add_header(
        "Content-Disposition", "attachment",
        filename=filename.encode("utf-8") if isinstance(filename, unicode)
        else filename)
    msg.attach(attachment)

    # 连接邮件服务
    server = smtplib.SMTP()
    server.connect('smtp.163.com')
    server.login(fromaddr, "FrrqWs5I")

    # 发送邮件
    text = msg.as_string()
    server.sendmail(fromaddr, toaddr, text)
    server.quit()
    
fromaddr = "18601036905@163.com"
senderName = u"杨乐"
# 多个收件人采用这种格式
toaddr = '"yangle@kaoshixing.com","766433030@qq.com"'
subject = u"报名统计表"
body = u"请查收附件"
# 附件名称
filename = "tsinghua1008.csv"
# 附件在磁盘上的路径
fileToSend = "/tmp/tsinghua1008.csv"

sendEmail(fromaddr, senderName, toaddr, subject, body, filename,
          fileToSend)
定时执行

采用crontab,执行crontab -e,加上:

0 12 8 10 * python test.py

表示10月8号的12点执行test.py脚本,crontab的用法此处不再赘述