Foxmail邮件会话模式实现逻辑

邮件信息的组成部分

  • folder: 位置信息收件箱 发件箱
  • Message_ID: 邮件的ID
  • subject: 主题
  • sessionSubject 会话主题。去除主题的前缀,如回复 Re 答复 自动回复 自动答复
  • References
    回复(全部回复)/转发一个邮件的时候,当前邮件回在头信息中添加References,参照Message_ID。多轮回复,邮件有多个References.Root引用在第一个位置

  • Default References

    (subject + 所有参与者)为组合生产一个定义的Default References,这个always有。

会话的聚合逻辑

聚合的原则:

将相同【会话主题】和具有【引用关系】邮件组成一个会话模式

* 引用关系包含邮件References和Default Reference

几种情况的说明:

  1. 无主题的不聚合

  2. 转发的虽然有引用,但是不聚合,从会话中脱离出来。因为主题发生了变化。“回复:你好”的会话主题是“你好”,而“转发:你好”的会话主题是“转发:你好”

具体步骤:

数据准备

  1. 拉取所有的收件箱邮件

  2. 按条件拉取发件箱,找跟收件箱的关系

    前提,必须 收件箱有的【会话主题】

    • 收件箱邮件引用
    • 引用收件箱
    • 收件箱有相同的邮件引用 & 收件时间 < 发件时间
    • 收件箱有相同的自定义引用 & 邮件引用不存在 & 收件时间 < 发件时间

Query.sql

SELECT
    SUBJECT,
    NAME,
    common_subject,
    message_id,
    session_id session_id,
    reference,
    send_time
FROM
    t_mail t
WHERE
    EXISTS (
        SELECT
            1
        FROM
            t_mail t2
        WHERE
            t. NAME = 'INBOX'  -- 【INBOX】所有邮件

            OR (    
                t.common_subject = t2.common_subject
                and (
                    (
                        t.message_id = t2.reference -- 被INBOX引用的【发件箱】邮件
                        AND t2.`name` = 'INBOX'
                    )
                    OR
                    (
                        t.session_id = t2.session_id -- INBOX时间之后,【发件箱】能组队的
                        AND t2.`name` = 'INBOX'
                        AND t.send_time > t2.send_time 
                        And t2.reference is null

                    )
                    or (
                        t.reference = t2.message_id
                        AND t2.`name` = 'INBOX'
                    )

                    OR (
                        t.reference = t2.reference -- INBOX时间之后,【发件箱】是引用的
                        AND t2.`name` = 'INBOX'
                        AND t.send_time > t2.send_time 
                    )
                )
            )
)

ORDER BY
    send_time DESC;
  1. 所有邮件按照发件顺序倒序排

数据聚合

遍历所有邮件

1. 有主题吗?没有主题自己单独一个会话。否则往下继续执行2

  1. 邮件引用,有没有引用其他邮件

    • 有引用,看有没有以【Reference ID+会话主题】为key的空间,如果有在尾部加入,没有自己开辟
    • 没有引用,继续3
  2. 邮件被引用

    • 有没有【Message_ID + 会话主题】的空间,有直接加入

– 没有继续详细执行4

  1. 自定义引用
    • 看有没有以【自定义引用+会话主题】为key的空间,如果有在尾部加入,没有自己开辟

foreach.java

list.forEach(mailEntity -> {

            if (StringUtils.isBlank(mailEntity.getCommonSubject())) { //无主题独占会话
                sessionMap.put(String.valueOf(IDUtils.genItemId()), Sets.newLinkedHashSet(mailEntity));
                return;
            }

            String reference = mailEntity.getReference();

            // 引用方
            if (reference != null) {
                Set<MailEntity> line = sessionMap.get(reference + ":" + mailEntity.getCommonSubject());

                if (line == null) {
                    line = Sets.newLinkedHashSet();
                    sessionMap.put(reference + ":" + mailEntity.getCommonSubject(), line);
                }

                line.add(mailEntity);
                return;
            }

            // 被引用方
            String messageId = mailEntity.getMessageId();

            Set<MailEntity> line = sessionMap.get(messageId + ":" + mailEntity.getCommonSubject());
            if (line != null) {
                line.add(mailEntity);
                return;
            }

            // 自建引用
            String sessionId = mailEntity.getSessionId();

            line = sessionMap.get(sessionId + ":" + mailEntity.getCommonSubject());

            if (line != null) {
                line.add(mailEntity);
                return;
            } else {
                line = Sets.newLinkedHashSet();
                sessionMap.put(sessionId + ":" + mailEntity.getCommonSubject(), line);
                line.add(mailEntity);
            }

        });

Docker下搭建FastDFS

1. 创建虚拟机,安装docker,docker-compose环境

2.创建目录和文件

$ mkdir -p /usr/local/docker/fastdfs
$ cd /usr/local/docker/fastdfs
$ vim docker-compose.yml
...

最终目录结构
image

2.1 docker-compose.yml

version: '3'

services:
 tracker:
  container_name: tracker
  image: "morunchang/fastdfs"
  network_mode: host
  entrypoint: sh tracker.sh
 storage0:
  container_name: storage0
  depends_on:
    - tracker
  build: ./storage
  network_mode: host
  environment:
   GROUP_NAME: storagegroup
   TRACKER_IP: 192.168.1.144:22122
  volumes:
    - /var/local/docker/fastdfs/storage0:/data/fast_data

2.2 storage/Dockerfile

Dockerfile

FROM morunchang/fastdfs
COPY nginx.conf /data/nginx/conf/nginx.conf
ENTRYPOINT sh storage.sh

2.2 storage/nginx.cnf

#user  nobody;
worker_processes  1;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;

    sendfile        on;

    keepalive_timeout  65;

    server {
        listen       80;
        server_name  localhost;

        location /storagegroup/M00 {
           proxy_next_upstream http_502 http_504 error timeout invalid_header;
             proxy_cache http-cache;
             proxy_cache_valid  200 304 12h;
             proxy_cache_key $uri$is_args$args;
             proxy_pass http://fdfs_group1;
             expires 30d;
         }

        location / {
            root   html;
            index  index.html index.htm;
        }

        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }

    }

}

启动

docker-compose up -d

客户端测试


参考网站
* https://blog.csdn.net/alinyua/article/details/82464496
* https://blog.csdn.net/lizhihaooo/article/details/79261388
* http://blog.csdn.net/ityouknow/article/details/79078175

注意如果拒绝访问,那么有可能是tracker的ip地址配置有误,或者防火墙需要关闭,关闭防火墙的参考https://www.cnblogs.com/moxiaoan/p/5683743.html

**外网部署,要开启23000端口

https://www.cnblogs.com/nbf-156cwl/articles/FastDFS.html

像outlook一样,做一个重复的周期性事件

项目有个需求,做个类似outlook的周期性会议的功能,如下图所示:

outlook会议周期性图片

  • 方案一:

    由于使用的是java平台,刚开始想通过成熟的调度框架quartz来实现,但是无法通过一个trigger实现一个复杂任务的调度。如:每2个月的第四周的周末,次行10次后结束

    quartz提供来四种计划:SimpleScheduleBuilder CronScheduleBuilder DailyTimeIntervalScheduleBuilder CalendarIntervalScheduleBuilder没种计划都有各自特定的场景。cronExpress也不是万能的,比如表示【每2个月的第四周的周末】是无法通过表达式:0 0 0 ?*/2 SAT#4,SUN#4,对于周来讲这样的写法会抛出异常,不运行逗号隔开多个

    Support for specifying multiple "nth" days is not implemented.
    
  • 方案二

    网上找了好久的方案,通过这篇文章日历设计之重复事件规则设计,知道了有个icalendar的协议RFC2445协议 (更加详细的中文解释在https://www.jianshu.com/p/8f8572292c58)。

    于是,我需要找基于java平台的支持RFC2445协议,找到了2个。

    1. lib-recur

    github地址:https://github.com/dmfs/lib-recur  \
    验证地址:http://recurrence-expansion-service.appspot.com/
    验证代码:

      RecurrenceRule rule = new RecurrenceRule("FREQ=MONTHLY;INTERVAL=1;BYDAY=2SA,2SU;COUNT=10");
    
        DateTime start = DateTime.parse("20180528T063302Z");//new DateTime(2018, Calendar.MAY /* 0-based month numbers! */,28, 14, 33, 02);
    
        RecurrenceRuleIterator it = rule.iterator(start);
    
        //        it.skip(0);
    
        int maxInstances = 20; // limit instances for rules that recur forever
    
        while (it.hasNext() && (!rule.isInfinite() || maxInstances-- > 0))
        {
            DateTime nextInstance = it.nextDateTime();
            System.out.println(nextInstance.toString());
            // do something with nextInstance
        }
    

    2. ical4j

    github地址:https://github.com/ical4j/ical4j

    代码演示:

     @Test
    public void testIcal4j() throws ParseException, IOException {
        DtStart dtstart = new DtStart("20180528T063302Z");
        DtStart endDate = new DtStart("20200925T063302Z");
    
        RRule rule = new RRule("FREQ=MONTHLY;INTERVAL=1;BYDAY=2SA,2SU;COUNT=10");
    
        VEvent sessionEvent = new VEvent(dtstart.getDate(), "hahahhehe");
    
        sessionEvent.getProperties().add(rule);
    //        sessionEvent.getProperties().add(dtstart);
        sessionEvent.getProperties().add(new Uid("2322332323323323"));
        sessionEvent.getProperties().add(new Location("南京堵路"));
    
        // 提醒,提前10分钟
        VAlarm valarm = new VAlarm(new Dur(0, 0, -10, 0));
        valarm.getProperties().add(new Summary("Event Alarm"));
        valarm.getProperties().add(Action.DISPLAY);
        valarm.getProperties().add(new Description("Progress Meeting at 9:30am"));
        sessionEvent.getAlarms().add(valarm);
    
    
        net.fortuna.ical4j.model.Calendar calendar = new net.fortuna.ical4j.model.Calendar();
        calendar.getProperties().add(new ProdId("-//Ben Fortuna//iCal4j 1.0//EN"));
        calendar.getProperties().add(Version.VERSION_2_0);
        calendar.getProperties().add(CalScale.GREGORIAN);
        calendar.getComponents().add(sessionEvent);
        calendar.validate();
    
    
    //        FileOutputStream fout = new FileOutputStream("/Users/rick/jkxyx205/log/2.ics");
    //        CalendarOutputter outputter = new CalendarOutputter();
    //        outputter.output(calendar, fout);
    
    //        PeriodList periodList = sessionEvent.getConsumedTime(dtstart.getDate(), endDate.getDate());
    //
    //        for(Period period : periodList) {
    //            System.out.println(period.get);
    //        }
    
    
        DateList list = rule.getRecur().getDates(dtstart.getDate(), dtstart.getDate(), endDate.getDate(), Value.DATE_TIME, 10);
    
        for(Date date : list) {
            System.out.println(date.toLocaleString());
        }
    }
    
    

    可以通过该库输出标准格式文件.ics,可参考文章ical4j 实现ICS文件的生成和解析

    为了提高性能,需要在客户端处理相关协议,基于javascript平台的支持RFC2445协议前端框架rrule

    github地址:https://github.com/jakubroztocil/rrule \
    验证地址:http://jakubroztocil.github.io/rrule/

更完整的rrule验证:https://www.textmagic.com/free-tools/rrule-generator

技术难点

    1. 用户在打开应用时, 当页面定位在某个月(周, 日)视图上时, 怎么样从这个用户中所有的周期事件中快速计算出所有的当前视图事件? 不可能每次请求一次, 全部事件计算一次.
    1. 日历的主要功能是提醒. 怎么样准时, 无误的把每分钟要提醒的短信, 邮件发出去, 这又是个量变到质变的难点, 如果当前数据库有几千万的周期事件?

后台中的Convert转换

分类

后台中的Convert,我总结了如下分类:

  1. springmvc自定义属性编辑器PropertyEditor

    我们在使用SpringMVC时,常常需要把表单中的参数映射到我们对象的属性中,我们只要使用默认的配置即可做到普通数据类型的转换,如将String转换成Integer和Double等。但如果我要把String映射到对象上,如“rick-18”映射到Student对象,就必须自定义converter。值得注意的是,这里的提交方式不能是raw(application/json)的 。只能是x-www-form-urlencoded/form-data类型

  2. springmvc自定义Converter

    Converter能完成属性编辑器的功能,而且更加通用,不仅仅可以接收UI参数。是Spring推荐的实现方式。关于两者的比较,可以查看https://stackoverflow.com/questions/12544479/spring-mvc-type-conversion-propertyeditor-or-converter

  3. stringmvc自定义HttpMessageConverter

    HTTP消息转换,比如可以让spring接收Content-Type:xx/yy类型,body是字符串“rick-18”,后台用对象Student接收。自定义raw。spring提供了很多默认的转换,如FormHttpMessageConverter,MappingJackson2HttpMessageConverter

  4. hibernate自定义属性

    hibernate在持久化,如果是自定义属性,该如何处理映射呢?如,有个属性是Student对象,希望能够将Student的json对象存储varchar到列stu上

  5. hibernate validation自定义验证方法

    如何自定义validation,验证逻辑错误。

实现

所有实现都是基于Springboot2.0,VEHICLE_BRAND#NS 映射到 Word

  • ## PropertyEditor
  1. 定义属性编辑器
    WordEditor.java
public class WordEditor extends PropertyEditorSupport {
    private static final String PARAM_SEPARATOR = "#";

    @Override
    public void setAsText(String text) throws IllegalArgumentException {
        if (StringUtils.hasText(text)) {
            String[] values = text.split(PARAM_SEPARATOR);


            Word word = new Word();
            word.setCategory(values[0]);
            word.setName(values[1]);
            setValue(word);

        } else {
            setValue(null);
        }
    }

    @Override
    public String getAsText() {
        Word word = (Word) getValue();
        if (word != null) {
            return word.getCategory() + "#" + word.getName();
        } else {
            return "";
        }
    }
}
  1. 全局绑定属性编辑器

PropertyConfiguration.java

@RestControllerAdvice
public class PropertyConfiguration {

    @InitBinder
    public void registerCustomEditors(WebDataBinder binder) {
        binder.registerCustomEditor(Word.class, new WordEditor());
    }
}

当然,可用在单独的Controller中绑定

  • ## Converter
  1. 定义Converter

StringToWordConverter.java

@Component
public class StringToWordConverter implements Converter<String, Word> {
    private static final String PARAM_SEPARATOR = "#";

    @Override
    public Word convert(String s) {
        String[] values = s.split(PARAM_SEPARATOR);


        Word word = new Word();
        word.setCategory(values[0]);
        word.setName(values[1]);
        return word;
    }
}

注意:Converter是来自接口org.springframework.core.convert.converter.Converter

  1. 添加Converter
@Configuration
public class MyWebMvcConfigurer implements WebMvcConfigurer {

    @Override
    public void addFormatters(FormatterRegistry registry) {
        registry.addConverter(new StringToWordConverter());
    }
}
  • ## HttpMessageConverter
  1. 定义 HttpMessageConverter

WordHttpMessageConverter.java

@Component
public class WordHttpMessageConverter extends AbstractHttpMessageConverter<Word> {
    private static final String PARAM_SEPARATOR = "#";

    public WordConverter() {
        super(new MediaType("text", "word"));
    }

    @Override
    protected boolean supports(Class<?> aClass) {
        return Word.class.isAssignableFrom(aClass);
    }

    @Override
    protected Word readInternal(Class<? extends Word> aClass, HttpInputMessage httpInputMessage) throws IOException, HttpMessageNotReadableException {
        String s = toString(httpInputMessage.getBody());

        String[] values = s.split(PARAM_SEPARATOR);


        Word word = new Word();
        word.setCategory(values[0]);
        word.setName(values[1]);
        return word;
    }

    @Override
    protected void writeInternal(Word word, HttpOutputMessage httpOutputMessage) throws IOException, HttpMessageNotWritableException {
        OutputStream outputStream = httpOutputMessage.getBody();
        String body = word.getCategory() + "#" + word.getName();
        outputStream.write(body.getBytes());
        outputStream.close();
    }

    private static String toString(InputStream inputStream) {
        Scanner scanner = new Scanner(inputStream, "UTF-8");
        return scanner.useDelimiter("\\A").next();
    }

}

前端可用通过Content-Type: text/word,进行消息转换

  1. 添加MessageConverter
@Configuration
public class OAWebMvcConfigurer implements WebMvcConfigurer {

    @Override
    public void extendMessageConverters(List<HttpMessageConverter<?>> converters) {
        converters.add(new WordConverter());
    }
}
  • ## hibernate Converter
  1. 新建Converter

WordJpaConverter.java

@Convert
public class WordJpaConverter implements AttributeConverter<Word, String> {

    private static final String PARAM_SEPARATOR = "#";

    @Override
    public String convertToDatabaseColumn(Word word) {
        if (Objects.isNull(word)) return "";

        StringBuilder sb = new StringBuilder();
        return sb.append(word.getCategory()).append(PARAM_SEPARATOR).append(word.getName()).toString();
    }

    @Override
    public Word convertToEntityAttribute(String s) {
        if (StringUtils.hasText(s)) {
            String[] values = s.split(PARAM_SEPARATOR);

            Word word = new Word();
            word.setCategory(values[0]);
            word.setName(values[1]);
            return word;

        }
        return null;
    }
} 
  1. 使用Converter

Vehicle.java

    @Entity
    public class Vehicle {
        @Id
        private Long id;

        @Convert(converter = WordJpaConverter.class)
        private Word vehicleBrand;
    }
  • ## hibernate validation
  1. 添加注解DictionaryConstraint.java
@Documented
@Constraint(validatedBy = DictionaryValidator.class)
@Target( { ElementType.METHOD, ElementType.FIELD })
@Retention(RetentionPolicy.RUNTIME)
public @interface DictionaryConstraint {
    String message() default "Invalid dictionary data";
    String name();
    Class<?>[] groups() default {};
    Class<? extends Payload>[] payload() default {};
}
  1. 设置验证逻辑DictionaryValidator.java
public class DictionaryValidator implements ConstraintValidator<DictionaryConstraint, Word> {

    private DictionaryConstraint constraintAnnotation;


    @Override
    public void initialize(DictionaryConstraint constraintAnnotation) {
        this.constraintAnnotation = constraintAnnotation;
    }

    @Override
    public boolean isValid(Word word, ConstraintValidatorContext constraintValidatorContext) {
        String category = constraintAnnotation.name();

        if (Objects.nonNull(category) && !Objects.equals(category, word.getCategory())) return false;

        DictionaryService dictionaryService = Global.applicationContext.getBean(DictionaryService.class);

        Word _word = dictionaryService.findByCategoryAndName(category, word.getName());

        return Objects.nonNull(_word);
    }
}
  1. 使用验证
    Vehicle.java
    @Entity
    public class Vehicle {
        @Id
        private Long id;

        @DictionaryConstraint(name = "VEHICLE_BRAND")
        @Convert(converter = WordJpaConverter.class)
        private Word vehicleBrand;
    }

前端数据的提交方式

提交数据body到后台有如下几种方式
* form-data
* x-www-form-urlencoded
* raw
– application/json
– application/xml
– text/plain
– text/html
– …
* binary

分类可用参考Postman

分类说明

Type 描述 example
x-www-form-urlencoded 一个普通的表单提交或者url地址栏参数提交;适用post,get方式 key=1&key=2
form-data 表单POST提交,设置enctype="multipart/form-data" POST / HTTP/1.1-----------------------------735323031399963166993862150Content-Disposition: form-data; name=”text1″
raw:application/json 通过ajax提交,Content-Type设置application/json,会绑定到Header中提交,这个类型可用自定义xx/yy,这需要后端的支持 {“name”:”rick”}

参考链接
https://unifaceinfo.com/docs/0907/Uniface_Library_HTML/ulibrary/MIMETypes_773A6E5F2234453F9A878E3961418FC0.html\
https://stackoverflow.com/questions/4526273/what-does-enctype-multipart-form-data-mean