郑州洛阳过周末

跟室友无聊想着周末找个地方去。郑州刚好可以周五晚上的火车去,周日晚上的火车回。用招行花费700元定了一晚郑州JW万豪,这就安排妥了。

火车大概晚上10点出发,第二天6点到达。虽然买的是软卧,但是这火车中途停站很多,基本没睡好。

火车站出来走19分钟即可到达二七广场。在附近逛逛,看到一居民小区的早餐店,吃了一碗胡辣汤。
胡辣汤
继续阅读“郑州洛阳过周末”

奥克兰体验生活一周


有好几个前同事到奥克兰工作了,所以我计划体验一下奥克兰的生活。 查看机票价格,海航往返含税2810元,深圳中转可以顺便回趟家,值得入手。行程如下:

2019.6.21 PEK-SZX
2019.6.22 SZX-AKL
2019.6.29 AKL-SZX
2019.7.01 SZX-PEK

在 Airbnb 预定了市中心的公寓,两个房间三张床,有客厅厨房,阳台可以看到海景。7晚连同手续费大概5800元人民币。大多数人来奥克兰只是中转,通常会转机到南岛。我们没车到处玩耍,就安心地在奥克兰体验生活吧。

买票的时候想着室友大概率请不到假,我很可能要一个人去,所以还买了我妈的票。在出发前,室友确认请到假了,于是变成了三个人的行程。 体验奥克兰家庭生活。
继续阅读“奥克兰体验生活一周”

hive failed renaming s3 table with error "New location for this table already exist"

Issue

- In hive-cli, rename table with command:

hive> alter table large_table_bk rename to large_table;

- 10 minutes later, it prompts error.

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Unable to alter table. New location for this table default.large_table already exists : s3://feichashao-hadoop/warehouse/large_table

- However, before executing the "rename" command, the directory was not exist in S3, so we don't expect such an error.
继续阅读“hive failed renaming s3 table with error "New location for this table already exist"”

hive-server2 remains 2k+ s3n-worker threads after job finished


tldr: This is a bug in EMR 5.6. Upgrading to EMR 5.8 or above can solve the issue.

Issue

User reported that he sees 2000+ s3n-worker threads after job finished. He has to restart the hive-server2 service everyday to mitigate the issue.

# sudo -u hive jstack 11089 | grep s3n-worker | wc -l
2000

The threads are repeating from s3n-worker-0 to s3n-worker-19. In another word, there are 100 * 20 s3n-worker threads.

"s3n-worker-19" #70 daemon prio=5 os_prio=0 tid=0x00007f5ac4cf0800 nid=0x10ad waiting on condition [0x00007f5ac1dee000]
......
"s3n-worker-1" #52 daemon prio=5 os_prio=0 tid=0x00007f5ac5462000 nid=0x109b waiting on condition [0x00007f5aca23f000]
"s3n-worker-0" #51 daemon prio=5 os_prio=0 tid=0x00007f5ac5480000 nid=0x109a waiting on condition [0x00007f5aca641000]
......

Environment

AWS EMR 5.6
继续阅读“hive-server2 remains 2k+ s3n-worker threads after job finished”