更新了部分文档
|
@ -1,10 +1,10 @@
|
|||
## Docker入门
|
||||
## Docker简易上手指南
|
||||
|
||||
### Docker简介
|
||||
|
||||
软件开发中最为麻烦的事情可能就是配置环境了。由于用户使用的操作系统具有多样性,即便使用跨平台的开发语言(如Java和Python)都不能保证代码能够在各种平台下都可以正常的运转,而且可能在不同的环境下我们的软件需要依赖的其他软件包也是不一样的。
|
||||
|
||||
那么问题来了,我们再安装软件的时候可不可以把软件运行的环境一并安装课?也就是说在安装软件的时候,我们是不是可以把原始环境一模一样地复制过来呢?
|
||||
那么问题来了,我们再安装软件的时候可不可以把软件运行的环境一并安装?也就是说在安装软件的时候,我们是不是可以把原始环境一模一样地复制过来呢?
|
||||
|
||||
虚拟机(virtual machine)就是带环境安装的一种解决方案,它可以在一种操作系统里面运行另一种操作系统,比如在Windows系统里面运行Linux系统,在macOS上运行Windows,而应用程序对此毫无感知。使用过虚拟机的人都知道,虚拟机用起来跟真实系统一模一样,而对于虚拟机的宿主系统来说,虚拟机就是一个普通文件,不需要了就删掉,对宿主系统或者其他的程序并没有影响。但是虚拟机通常会占用较多的系统资源,启动和关闭也非常的缓慢,总之用户体验没有想象中的那么好。
|
||||
|
||||
|
@ -87,7 +87,7 @@ docker stop <container-id>
|
|||
docker stop <name>
|
||||
```
|
||||
|
||||
对于那些不会自动终止的容器,就可以用下面的方式来停止。
|
||||
对于那些不会自动终止的容器,就可以用下面的方式来停止。
|
||||
|
||||
```Shell
|
||||
docker container kill <container-id>
|
||||
|
@ -107,10 +107,10 @@ service docker start
|
|||
|
||||
```JavaScript
|
||||
{
|
||||
"registry-mirrors": [
|
||||
"http://hub-mirror.c.163.com",
|
||||
"https://registry.docker-cn.com"
|
||||
]
|
||||
"registry-mirrors": [
|
||||
"http://hub-mirror.c.163.com",
|
||||
"https://registry.docker-cn.com"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
|
@ -195,3 +195,4 @@ select user, host, plugin, authentication_string from user where user='root';
|
|||
2 rows in set (0.00 sec)
|
||||
```
|
||||
|
||||
接下来就已经可以访问你的MySQL服务器啦,当然远程连接的时候不要忘了在防火墙上开启对应的端口。
|
After Width: | Height: | Size: 35 KiB |
After Width: | Height: | Size: 48 KiB |
After Width: | Height: | Size: 63 KiB |
After Width: | Height: | Size: 110 KiB |
After Width: | Height: | Size: 113 KiB |
After Width: | Height: | Size: 36 KiB |
After Width: | Height: | Size: 22 KiB |
After Width: | Height: | Size: 184 KiB |
After Width: | Height: | Size: 313 KiB |
After Width: | Height: | Size: 195 KiB |
After Width: | Height: | Size: 209 KiB |
After Width: | Height: | Size: 86 KiB |
After Width: | Height: | Size: 140 KiB |
After Width: | Height: | Size: 58 KiB |
After Width: | Height: | Size: 400 KiB |
After Width: | Height: | Size: 218 KiB |
After Width: | Height: | Size: 290 KiB |
After Width: | Height: | Size: 51 KiB |
After Width: | Height: | Size: 104 KiB |
After Width: | Height: | Size: 253 KiB |
After Width: | Height: | Size: 142 KiB |
After Width: | Height: | Size: 158 KiB |
After Width: | Height: | Size: 13 KiB |
After Width: | Height: | Size: 133 KiB |
After Width: | Height: | Size: 147 KiB |
After Width: | Height: | Size: 189 KiB |
After Width: | Height: | Size: 178 KiB |
After Width: | Height: | Size: 86 KiB |
After Width: | Height: | Size: 27 KiB |
After Width: | Height: | Size: 1.0 MiB |
After Width: | Height: | Size: 5.9 KiB |
After Width: | Height: | Size: 6.4 KiB |
After Width: | Height: | Size: 148 KiB |
After Width: | Height: | Size: 296 KiB |
After Width: | Height: | Size: 82 KiB |
After Width: | Height: | Size: 726 KiB |
After Width: | Height: | Size: 345 KiB |
After Width: | Height: | Size: 44 KiB |
After Width: | Height: | Size: 298 KiB |
After Width: | Height: | Size: 212 KiB |
After Width: | Height: | Size: 238 KiB |
After Width: | Height: | Size: 212 KiB |
After Width: | Height: | Size: 72 KiB |
After Width: | Height: | Size: 146 KiB |
After Width: | Height: | Size: 439 KiB |
After Width: | Height: | Size: 37 KiB |
After Width: | Height: | Size: 22 KiB |
After Width: | Height: | Size: 174 KiB |
After Width: | Height: | Size: 48 KiB |
|
@ -1,4 +1,4 @@
|
|||
## 项目实战 - 开启团队项目
|
||||
## 开启团队项目
|
||||
|
||||
### 创建项目
|
||||
|
|
@ -0,0 +1,398 @@
|
|||
## 电商网站技术要点剖析
|
||||
|
||||
### 商业模式
|
||||
|
||||
1. B2B - 商家对商家,交易双方都是企业(商家),最典型的案例就是阿里巴巴。
|
||||
2. C2C - 个人对个人,例如:淘宝、人人车。
|
||||
3. B2C - 商家对个人,例如:唯品会,聚美优品。
|
||||
4. C2B - 个人对商家,先有消费者提出需求,后有商家按需求组织生产,例如: 尚品宅配。
|
||||
5. O2O - 线上到线下,将线下的商务机会与互联网结合,让互联网成为线下交易的平台,例如:美团外卖、饿了么。
|
||||
6. B2B2C - 商家对商家对个人,例如:天猫、京东。
|
||||
|
||||
### 需求要点
|
||||
|
||||
1. 用户端
|
||||
- 首页(商品分类、广告轮播、滚动快讯、瀑布加载)
|
||||
|
||||
- 用户(登录(第三方登录)、注册、注销、自服务(个人信息、浏览历史、收货地址、……))
|
||||
|
||||
- 商品(分类、列表、详情、搜索、添加到购物车)
|
||||
- 购物车(查看、编辑(修改数量、删除商品、清空))
|
||||
- 订单(提交订单(支付)、历史订单、订单详情、订单评价)
|
||||
2. 管理端
|
||||
|
||||
> 提示:可以通过思维导图来进行需求的整理,思维导图上的每个叶子节点都是不可再拆分的功能。
|
||||
|
||||
### 物理模型设计
|
||||
|
||||
两个概念:SPU(Standard Product Unit)和SKU(Stock Keeping Unit)。
|
||||
|
||||
- SPU:iPhone 6s
|
||||
- SKU:iPhone 6s 64G 土豪金
|
||||
|
||||
![](./res/shopping-pdm.png)
|
||||
|
||||
### 第三方登录
|
||||
|
||||
第三方登录是指利用第三方网站(通常是知名社交网站)的账号进行登录验证,比如国内的 QQ、微博,国外的Google、Facebook等,第三方登录大部分都是使用[OAuth](),它是一个关于授权的开放网络标准,得到了广泛的应用,目前通常使用的是2.0版本。关于OAuth的基础知识,可以阅读阮一峰老师的[《理解OAuth 2.0》](http://www.ruanyifeng.com/blog/2014/05/oauth_2_0.html)。
|
||||
|
||||
#### OAuth 2.0授权流程
|
||||
|
||||
1. 用户打开客户端以后,客户端要求用户(资源所有者)给予授权。
|
||||
2. 用户(资源所有者)同意给予客户端授权。
|
||||
3. 客户端使用上一步获得的授权,向认证服务器申请访问令牌。
|
||||
4. 认证服务器对客户端进行认证以后,发放访问令牌。
|
||||
5. 客户端使用访问令牌向资源服务器申请获取资源。
|
||||
6. 资源服务器确认访问令牌无误,同意向客户端开放资源。
|
||||
|
||||
![](./res/oauth2.png)
|
||||
|
||||
如果使用微博登录进行接入,其具体步骤可以参考微博开放平台上的[“微博登录接入”](http://open.weibo.com/wiki/Connect/login)文档。使用QQ登录进行接入,需要首先注册成为QQ互联开发者并通过审核,具体的步骤可以参考QQ互联上的[“接入指南”](http://wiki.connect.qq.com/),具体的步骤可以参考[“网站开发流程”](http://wiki.connect.qq.com/%E5%87%86%E5%A4%87%E5%B7%A5%E4%BD%9C_oauth2-0)。
|
||||
|
||||
> 提示:在Gitbook上面有一本名为[《Django博客入门》](https://shenxgan.gitbooks.io/django/content/publish/2015-08-10-django-oauth-login.html)的书以Github为例介绍了第三方账号登录,有兴趣的可以自行阅读。
|
||||
|
||||
通常电商网站在使用第三方登录时,会要求与网站账号进行绑定或者根据获取到的第三方账号信息(如:手机号)自动完成账号绑定。
|
||||
|
||||
### 缓存预热和查询缓存
|
||||
|
||||
#### 缓存预热
|
||||
|
||||
所谓缓存预热,是指在启动服务器时将数据提前加载到缓存中,为此可以在Django应用的`apps.py`模块中编写`AppConfig`的子类并重写`ready()`方法,代码如下所示。
|
||||
|
||||
```Python
|
||||
import pymysql
|
||||
|
||||
from django.apps import AppConfig
|
||||
from django.core.cache import cache
|
||||
|
||||
SELECT_PROVINCE_SQL = 'select distid, name from tb_district where pid is null'
|
||||
|
||||
|
||||
class CommonConfig(AppConfig):
|
||||
name = 'common'
|
||||
|
||||
def ready(self):
|
||||
conn = pymysql.connect(host='1.2.3.4', port=3306,
|
||||
user='root', password='pass',
|
||||
database='db', charset='utf8',
|
||||
cursorclass=pymysql.cursors.DictCursor)
|
||||
try:
|
||||
with conn.cursor() as cursor:
|
||||
cursor.execute(SELECT_PROVINCE_SQL)
|
||||
provinces = cursor.fetchall()
|
||||
cache.set('provinces', provinces)
|
||||
finally:
|
||||
conn.close()
|
||||
```
|
||||
|
||||
接下来,还需要在应用的`__init__.py`中编写下面的代码。
|
||||
|
||||
```Python
|
||||
default_app_config = 'common.apps.CommonConfig'
|
||||
```
|
||||
|
||||
或者在项目的`settings.py`文件中注册应用。
|
||||
|
||||
```Python
|
||||
INSTALLED_APPS = [
|
||||
...
|
||||
'common.apps.CommonConfig',
|
||||
...
|
||||
]
|
||||
```
|
||||
|
||||
#### 查询缓存
|
||||
|
||||
```Python
|
||||
from pickle import dumps
|
||||
from pickle import loads
|
||||
|
||||
from django.core.cache import caches
|
||||
|
||||
MODEL_CACHE_KEY = 'project:modelcache:%s'
|
||||
|
||||
|
||||
def my_model_cache(key, section='default', timeout=None):
|
||||
"""实现模型缓存的装饰器"""
|
||||
|
||||
def wrapper1(func):
|
||||
|
||||
def wrapper2(*args, **kwargs):
|
||||
real_key = '%s:%s' % (MODEL_CACHE_KEY % key, ':'.join(map(str, args)))
|
||||
serialized_data = caches[section].get(real_key)
|
||||
if serialized_data:
|
||||
data = loads(serialized_data)
|
||||
else:
|
||||
data = func(*args, **kwargs)
|
||||
cache.set(real_key, dumps(data), timeout=timeout)
|
||||
return data
|
||||
|
||||
return wrapper2
|
||||
|
||||
return wrapper1
|
||||
```
|
||||
|
||||
```Python
|
||||
@my_model_cache(key='provinces')
|
||||
def get_all_provinces():
|
||||
return list(Province.objects.all())
|
||||
```
|
||||
|
||||
### 购物车实现
|
||||
|
||||
问题一:已登录用户的购物车放在哪里?未登录用户的购物车放在哪里?
|
||||
|
||||
```Python
|
||||
class CartItem(object):
|
||||
"""购物车中的商品项"""
|
||||
|
||||
def __init__(self, sku, amount=1, selected=False):
|
||||
self.sku = sku
|
||||
self.amount = amount
|
||||
self.selected = selected
|
||||
|
||||
@property
|
||||
def total(self):
|
||||
return self.sku.price * self.amount
|
||||
|
||||
|
||||
class ShoppingCart(object):
|
||||
"""购物车"""
|
||||
|
||||
def __init__(self):
|
||||
self.items = {}
|
||||
self.index = 0
|
||||
|
||||
def add_item(self, item):
|
||||
if item.sku.id in self.items:
|
||||
self.items[item.sku.id].amount += item.amount
|
||||
else:
|
||||
self.items[item.sku.id] = item
|
||||
|
||||
def remove_item(self, sku_id):
|
||||
if sku_id in self.items:
|
||||
self.items.remove(sku_id)
|
||||
|
||||
def clear_all_items(self):
|
||||
self.items.clear()
|
||||
|
||||
@property
|
||||
def cart_items(self):
|
||||
return self.items.values()
|
||||
|
||||
@property
|
||||
def cart_total(self):
|
||||
total = 0
|
||||
for item in self.items.values():
|
||||
total += item.total
|
||||
return total
|
||||
```
|
||||
|
||||
已登录用户的购物车可以放在数据库中(可以先在Redis中缓存);未登录用户的购物车可以保存在Cookie中(减少服务器端内存开销)。
|
||||
|
||||
```JSON
|
||||
{
|
||||
'1001': {sku: {...}, 'amount': 1, 'selected': True},
|
||||
'1002': {sku: {...}, 'amount': 2, 'selected': False},
|
||||
'1003': {sku: {...}, 'amount': 3, 'selected': True},
|
||||
}
|
||||
```
|
||||
|
||||
```Python
|
||||
request.get_signed_cookie('cart')
|
||||
|
||||
cart_base64 = base64.base64encode(pickle.dumps(cart))
|
||||
response.set_signed_cookie('cart', cart_base64)
|
||||
```
|
||||
|
||||
问题二:用户登录之后,如何合并购物车?(目前电商应用的购物车几乎都做了持久化处理,主要是方便在多个终端之间共享数据)
|
||||
|
||||
### 集成支付功能
|
||||
|
||||
问题一:支付信息如何持久化?(必须保证每笔交易都有记录)
|
||||
|
||||
问题二:如何接入支付宝?(接入其他平台基本类似)
|
||||
|
||||
1. [蚂蚁金服开放平台](https://open.alipay.com/platform/home.htm)。
|
||||
2. [入驻平台](https://open.alipay.com/platform/homeRoleSelection.htm)。
|
||||
3. [开发者中心](https://openhome.alipay.com/platform/appManage.htm#/apps)。
|
||||
4. [文档中心](https://docs.open.alipay.com/270/105899/)。
|
||||
5. [SDK集成](https://docs.open.alipay.com/54/103419) - [PYPI链接](https://pypi.org/project/alipay-sdk-python/)。
|
||||
6. [API列表](https://docs.open.alipay.com/270/105900/)。
|
||||
|
||||
![](./res/alipay_web_developer.png)
|
||||
|
||||
配置文件:
|
||||
|
||||
```Python
|
||||
ALIPAY_APPID = '......'
|
||||
ALIPAY_URL = 'https://openapi.alipaydev.com/gateway.do'
|
||||
ALIPAY_DEBUG = False
|
||||
```
|
||||
|
||||
获得支付链接(发起支付):
|
||||
|
||||
```Python
|
||||
# 创建调用支付宝的对象
|
||||
alipay = AliPay(
|
||||
# 在线创建应用时分配的ID
|
||||
appid=settings.ALIPAY_APPID,
|
||||
app_notify_url=None,
|
||||
# 自己应用的私钥
|
||||
app_private_key_path=os.path.join(
|
||||
os.path.dirname(os.path.abspath(__file__)),
|
||||
'keys/app_private_key.pem'),
|
||||
# 支付宝的公钥
|
||||
alipay_public_key_path=os.path.join(
|
||||
os.path.dirname(os.path.abspath(__file__)),
|
||||
'keys/alipay_public_key.pem'),
|
||||
sign_type='RSA2',
|
||||
debug=settings.ALIPAY_DEBUG
|
||||
)
|
||||
# 调用获取支付页面操作
|
||||
order_info = alipay.api_alipay_trade_page_pay(
|
||||
out_trade_no='...',
|
||||
total_amount='...',
|
||||
subject='...',
|
||||
return_url='http://...'
|
||||
)
|
||||
# 生成完整的支付页面URL
|
||||
alipay_url = settings.ALIPAY_URL + '?' + order_info
|
||||
return JsonResponse({'alipay_url': alipay_url})
|
||||
```
|
||||
|
||||
通过上面返回的链接可以进入支付页面,支付完成后会自动跳转回上面代码中设定好的项目页面,在该页面中可以获得订单号(out_trade_no)、支付流水号(trade_no)、交易金额(total_amount)和对应的签名(sign)并请求后端验证和保存交易结果,代码如下所示:
|
||||
|
||||
```Python
|
||||
# 创建调用支付宝的对象
|
||||
alipay = AliPay(
|
||||
# 在线创建应用时分配的ID
|
||||
appid=settings.ALIPAY_APPID,
|
||||
app_notify_url=None,
|
||||
# 自己应用的私钥
|
||||
app_private_key_path=os.path.join(
|
||||
os.path.dirname(os.path.abspath(__file__)),
|
||||
'keys/app_private_key.pem'),
|
||||
# 支付宝的公钥
|
||||
alipay_public_key_path=os.path.join(
|
||||
os.path.dirname(os.path.abspath(__file__)),
|
||||
'keys/alipay_public_key.pem'),
|
||||
sign_type='RSA2',
|
||||
debug=settings.ALIPAY_DEBUG
|
||||
)
|
||||
# 请求参数(假设是POST请求)中包括订单号、支付流水号、交易金额和签名
|
||||
params = request.POST.dict()
|
||||
# 调用验证操作
|
||||
if alipay.verify(params, params.pop('sign')):
|
||||
# 对交易进行持久化操作
|
||||
```
|
||||
|
||||
支付宝的支付API还提供了交易查询、交易结算、退款、退款查询等一系列的接口,可以根据业务需要进行调用,此处不再进行赘述。
|
||||
|
||||
### 秒杀和超卖
|
||||
|
||||
1. 秒杀:秒杀是通常意味着要在很短的时间处理极高的并发,系统在短时间需要承受平时百倍以上的流量,因此秒杀架构是一个比较复杂的问题,其核心思路是流量控制和性能优化,需要从前端(通过JavaScript实现倒计时、避免重复提交和限制频繁刷新)到后台各个环节的配合。流量控制主要是限制只有少部分流量进入服务后端(毕竟最终只有少部分用户能够秒杀成功),同时在物理架构上使用缓存(一方面是因为读操作多写操作少;另外可以将库存放在Redis中,利用DECR原语实现减库存;同时也可以利用Redis来进行限流,道理跟限制频繁发送手机验证码是一样的)和消息队列(消息队列最为重要的作用就是“削峰”和“上下游节点解耦合”)来进行优化;此外还要采用无状态服务设计,这样才便于进行水平扩展(通过增加设备来为系统扩容)。
|
||||
2. 超卖现象:比如某商品的库存为1,此时用户1和用户2并发购买该商品,用户1提交订单后该商品的库存被修改为0,而此时用户2并不知道的情况下提交订单,该商品的库存再次被修改为-1这就是超卖现象。解决超卖现象有三种常见的思路:
|
||||
- 悲观锁控制:查询商品数量的时候就用`select ... for update`对数据加锁,这样的话用户1查询库存时,用户2因无法读取库存数量被阻塞,直到用户1提交或者回滚了更新库存的操作后才能继续,从而解决了超卖问题。但是这种做法对并发访问量很高的商品来说性能太过糟糕,实际开发中可以在库存小于某个值时才考虑加锁,但是总的来说这种做法不太可取。
|
||||
- 乐观锁控制:查询商品数量不用加锁,更新库存的时候设定商品数量必须与之前查询数量相同才能更新,否则说明其他事务已经更新了库存,必须重新发出请求。这种做法要求事务隔离级别为可重复读,否则仍然会产生问题。
|
||||
- 尝试减库存:将上面的查询(`select`)和更新(`update`)操作合并为一条SQL操作,更新库存的时候,在`where`筛选条件中加上`库存>=购买数量`或`库存-购买数量>=0`的条件。
|
||||
|
||||
> 提示:有兴趣的可以自己在知乎上看看关于这类问题的讨论。
|
||||
|
||||
### 静态资源管理
|
||||
|
||||
静态资源的管理可以自己架设文件服务器或者分布式文件服务器(FastDFS),但是一般的项目中没有必要这样做而且效果未必是最好的,我们建议使用云存储服务来管理网站的静态资源,国内外的云服务提供商如亚马逊、阿里云、腾讯云、七牛、LeanCloud、Bmob等都提供了非常优质的云存储服务,而且价格也是一般公司可以接受的。可以参考《在阿里云OSS上托管静态网站》一文来完成对网站静态资源的管理,代码相关的内容可以参考阿里云的[对象存储 OSS开发人员指南](https://www.alibabacloud.com/zh/support/developer-resources)。
|
||||
|
||||
### 全文检索
|
||||
|
||||
#### 方案选择
|
||||
|
||||
1. 使用数据库的模糊查询功能 - 效率低,每次需要全表扫描,不支持分词。
|
||||
2. 使用数据库的全文检索功能 - MySQL 5.6以前只适用于MyISAM引擎,检索操作和其他的DML操作耦合在数据库中,可能导致检索操作非常缓慢,数据量达到百万级性能显著下降,查询时间很长。
|
||||
3. 使用开源搜索引擎 - 索引数据和原始数据分离,可以使用ElasticSearch或Solr来提供外置索引服务,如果不考虑高并发的全文检索需求,纯Python的Whoosh也可以考虑。ElasticSearch-Analysis-IK
|
||||
|
||||
#### ElasticSearch
|
||||
|
||||
ElasticSearch是一个高可扩展的开源全文搜索和分析引擎,它允许存储、搜索和分析大量的数据,并且这个过程是近实时的。它通常被用作底层引擎和技术,为复杂的搜索功能和要求提供动力,大家熟知的维基百科、Stack-Overflow、Github都使用了ElasticSearch。
|
||||
|
||||
ElasticSearch的底层是开源搜索引擎[Lucene](https://lucene.apache.org/)。但是直接用Lucene会非常麻烦,必须自己编写代码去调用它的接口而且只支持Java语言。ElasticSearch相当于是对Lucene进行了封装,提供了REST API的操作接口。搜索引擎在对数据构建索引时,需要进行分词处理。ElasticSearch不支持基于中文分词的索引,需要配合扩展elasticsearch-analysis-ik来实现中文分词处理。
|
||||
|
||||
ElasticSearch的安装和配置可以参考[《ElasticSearch之Docker安装》](https://blog.csdn.net/jinyidong/article/details/80475320)。除了ElasticSearch之外,也可以使用Solr、Whoosh等来提供搜索引擎服务,基本上Django项目中可以考虑如下两套方案:
|
||||
|
||||
- Haystack(django-haystack / drf-haystack) + Whoosh + Jieba
|
||||
|
||||
- Haystack (django-haystack / drf-haystack)+ ElasticSearch + ElasticSearch-Analysis-IK
|
||||
|
||||
#### Django对接ElasticSearch
|
||||
|
||||
Python对接ElasticSearch的第三方库是HayStack,在Django项目中可以使用django-haystack,通过HayStack可以在不修改代码对接多种搜索引擎服务。
|
||||
|
||||
```shell
|
||||
pip install django-haystack elasticsearch
|
||||
```
|
||||
|
||||
配置文件:
|
||||
|
||||
```Python
|
||||
INSTALLED_APPS = [
|
||||
...
|
||||
'haystack',
|
||||
...
|
||||
]
|
||||
|
||||
HAYSTACK_CONNECTIONS = {
|
||||
'default': {
|
||||
# 引擎配置
|
||||
'ENGINE':
|
||||
'haystack.backends.elasticsearch_backend.ElasticsearchSearchEngine',
|
||||
# 搜索引擎服务的URL
|
||||
'URL': 'http://...',
|
||||
# 索引库的名称
|
||||
'INDEX_NAME': '...',
|
||||
},
|
||||
}
|
||||
|
||||
# 添加/删除/更新数据时自动生成索引
|
||||
HAYSTACK_SIGNAL_PROCESSOR = 'haystack.signals.RealtimeSignalProcessor'
|
||||
```
|
||||
|
||||
索引类:
|
||||
|
||||
```Python
|
||||
from haystack import indexes
|
||||
|
||||
|
||||
class HouseInfo(indexes.SearchIndex, indexes.Indexable):
|
||||
|
||||
text = indexes.CharField(document=True, use_template=True)
|
||||
|
||||
def get_model(self):
|
||||
return HouseInfo
|
||||
|
||||
def index_queryset(self, using=None):
|
||||
return self.get_model().objects.all()
|
||||
```
|
||||
|
||||
编辑text字段的模板(需要放在templates/search/indexes/rent/houseinfo_text.txt):
|
||||
|
||||
```
|
||||
{{object.title}}
|
||||
{{object.detail}}
|
||||
```
|
||||
|
||||
配置URL:
|
||||
|
||||
```Python
|
||||
urlpatterns = [
|
||||
# ...
|
||||
url('search/', include('haystack.urls')),
|
||||
]
|
||||
```
|
||||
|
||||
生成初始索引:
|
||||
|
||||
```Shell
|
||||
python manage.py rebuild_index
|
||||
```
|
||||
|
||||
> 说明:可以参考[《Django Haystack 全文检索与关键词高亮》](https://www.zmrenwu.com/post/45/)一文来更深入的了解基于Haystack的全文检索操作。
|
|
@ -61,32 +61,33 @@ API接口返回的数据通常都是JSON或XML格式,我们这里不讨论后
|
|||
请求参数:
|
||||
|
||||
| 参数名 | 类型 | 是否必填 | 参数位置 | 说明 |
|
||||
| ------ | ------ | -------- | -------- | ------------------------------------ | |
|
||||
| ------ | ------ | -------- | -------- | ------------------------------------ |
|
||||
| page | 整数 | 否 | 查询参数 | 页码,默认值1 |
|
||||
| size | 整数 | 否 | 查询参数 | 每次获取评论数量(10~100),默认值20 |
|
||||
| key | 字符串 | 是 | 请求头 | 用户的身份标识 |
|
||||
|
||||
响应信息:
|
||||
|
||||
```JSON
|
||||
|
||||
{
|
||||
"code": 10000,
|
||||
"message": "获取评论成功",
|
||||
"page": 1,
|
||||
"size": 10,
|
||||
"totalPage": 35,
|
||||
"contents": [
|
||||
{
|
||||
"userId": 1700095,
|
||||
"nickname": "王大锤",
|
||||
"pubDate": "2018年7月31日",
|
||||
"content": "小编是不是有病呀"
|
||||
"content": "小编是不是有病呀",
|
||||
/* ... */
|
||||
},
|
||||
{
|
||||
"userId", 1995322,
|
||||
"nickname": "白元芳",
|
||||
"pubDate": "2018年8月2日",
|
||||
"content": "楼上说得好"
|
||||
"content": "楼上说得好",
|
||||
/* ... */
|
||||
}
|
||||
]
|
||||
|
@ -108,16 +109,15 @@ API接口返回的数据通常都是JSON或XML格式,我们这里不讨论后
|
|||
|
||||
请求参数:
|
||||
|
||||
| 参数名 | 类型 | 是否必填 | 参数位置 | 说明 |
|
||||
| ------- | ------ | -------- | -------- | ------------------------------------ |
|
||||
| userId | 字符串 | 是 | 消息体 | 用户ID |
|
||||
| token | 字符串 | 是 | 消息体 | 用户的令牌或URL的签名 |
|
||||
| content | 字符串 | 是 | 消息体 | 评论的内容 |
|
||||
| 参数名 | 类型 | 是否必填 | 参数位置 | 说明 |
|
||||
| ------- | ------ | -------- | -------- | ---------- |
|
||||
| userId | 字符串 | 是 | 消息体 | 用户ID |
|
||||
| key | 字符串 | 是 | 请求头 | 用户的令牌 |
|
||||
| content | 字符串 | 是 | 消息体 | 评论的内容 |
|
||||
|
||||
响应信息:
|
||||
|
||||
```JSON
|
||||
|
||||
{
|
||||
"code": 10001,
|
||||
"message": "创建评论成功",
|
||||
|
@ -129,5 +129,3 @@ API接口返回的数据通常都是JSON或XML格式,我们这里不讨论后
|
|||
/* ... */
|
||||
}
|
||||
```
|
||||
|
||||
|
|
@ -0,0 +1,89 @@
|
|||
## 英语面试
|
||||
|
||||
以下用I表示面试官(Interviewer),用C表示面试者(Candidate)。
|
||||
|
||||
### 开场寒暄
|
||||
|
||||
1. I: Thanks for waiting. (Please follow me.)
|
||||
|
||||
C: It's no problem.
|
||||
|
||||
2. I: How are you doing this morning?
|
||||
|
||||
C: I'm great. / I'm doing fine. Thank you. / How about you?
|
||||
|
||||
3. I: How did you get here?
|
||||
|
||||
C: I took the subway here. / I drove here.
|
||||
|
||||
4. I: Glad to meet you.
|
||||
|
||||
C: Glad to meet you. / It's great to finally meet you in person. (之前电话沟通过的)
|
||||
|
||||
### 正式面试
|
||||
|
||||
#### 人力面试
|
||||
|
||||
1. I: Can you tell me a little bit about yourself? (介绍下自己)
|
||||
|
||||
原则:不要谈私生活和奇怪的癖好(英雄联盟干到钻石),因为别人更想知道的是你的专业技能(qulifications)和工作经验(experience),所以重点在你之前的公司(company name)、职位(title)、时间(years)和主要职责(major responsibilities)
|
||||
|
||||
C: Thank you for having me. My name is Dachui WANG. I'm 25 years old, and I'm single. I have a Bachelor's Degree of Computer Science from Tsinghua University. I was a Junior Java Programmer for ABC Technologies during my college life. Then I become an intermediate Java engineer for XYZ Corporation in last two years. Programming is my everyday life and programming is where my passion is. I think I have a good knowledge of Java enterprise application developement using light-weight frameworks like Spring, Guice, Hibernate and other open source middle-ware like Dubbo, Mycat, rocketmq and so on and so forth. I love reading, travelling and playing basketball in my spare time. That's all! Thank you!
|
||||
|
||||
2. I: How would you describe your personality? (你的性格)
|
||||
|
||||
C: I'm hard working, eager to learn, and very serious about my work. I enjoy working with other people and I love challenges.
|
||||
|
||||
3. I: What do you know about our company? (你对我们公司有什么了解)
|
||||
|
||||
(需要做功课,了解公司的状况和企业文化,该公司在这个行业中的一个状况,有什么核心业务,主要的竞争对手有哪些)
|
||||
|
||||
C: The one thing that I like the most about our company is your core values. I think they're very important in this industry because …(自由发挥的部分)... I personally really believe in the cause as well. Of course, I'm very interested in your products such as …(功课部分)… and the techniques behind them.
|
||||
|
||||
4. I: Why are you leaving your last job? (为什么离职)
|
||||
|
||||
C: I want to advance my career and I think this job offers more challenges and opportunities for me do to that.
|
||||
|
||||
5. I: What do you see yourself in 3 or 5 years? (3-5年职业规划)
|
||||
|
||||
C: My long term goals involve growing with the company, where I can continue to learn, to take on additional responsibilities and to contribute as much value as I can. I intend to take advantage of all of these.
|
||||
|
||||
6. I: What's your salary expectation? (期望薪资)
|
||||
|
||||
C: My salary expectation is in line with my experience and qualifications. I believe our company will pay me and every other employee fairly. (把球踢给对方先看看对方报价是多少,如果对方非要你报价再说后面的内容) I think 15 thousands RMB or above is fitting for me to leave in Chengdu.
|
||||
|
||||
7. I: Do you have any questions for me? (问面试官的问题)
|
||||
|
||||
C: What's the growth potential for this position?
|
||||
|
||||
|
||||
#### 技术面试
|
||||
|
||||
1. I: What's difference between an interface and an abstract class?
|
||||
2. I: What are pass by reference and pass by value?
|
||||
3. I: What's the difference between process and threads?
|
||||
4. I: Explain the available thread state in high-level.
|
||||
5. I: What's deadlocks? How to avoid them?
|
||||
6. I: How HashMap works in Java?
|
||||
7. I: What's the difference between ArrayList and LinkedList? (类似的问题还有很多,比如比较HashSet和TreeSet、HashMap和Hashtable)
|
||||
8. I: Tell me what you know about garbage collection in Java.
|
||||
9. I: What're two types of exceptions in Java?
|
||||
10. I: What's the advantage of PreparedStatement over Statement?
|
||||
11. I: What's the use of CallableStatement?
|
||||
12. I: What does connection pool mean?
|
||||
13. I: Explain the life cycle of a Servlet.
|
||||
14. I: What's the difference between redirect and forward?
|
||||
15. I: What's EL? What're implicit objects of EL?
|
||||
16. I: Tell me what you know about Spring framework and its benefits.
|
||||
17. I: What're different types of dependency injection.
|
||||
18. I: Are singleton beans thread safe in Spring framework?
|
||||
19. I: What're the benefits of Spring framework's transaction management?
|
||||
20. I: Explain what's AOP.
|
||||
21. I: What's a proxy and how to implement proxy pattern?
|
||||
22. I: How Spring MVC works?
|
||||
23. I: What's the working scenario of Hibernate and MyBatis?
|
||||
24. I: How to implement SOA?
|
||||
25. I: Make a brief introduction of the projects you are involved before?
|
||||
|
||||
|
||||
上面主要是面试Java程序员的问题,但是整个流程大致如此。
|
|
@ -0,0 +1,90 @@
|
|||
## 面试中的公共问题
|
||||
|
||||
### 计算机基础
|
||||
|
||||
1. TCP/IP模型相关问题。
|
||||
|
||||
> 建议阅读阮一峰的[《互联网协议入门(一)》](http://www.ruanyifeng.com/blog/2012/05/internet_protocol_suite_part_i.html)和[《互联网协议入门(二)》](http://www.ruanyifeng.com/blog/2012/06/internet_protocol_suite_part_ii.html)。
|
||||
|
||||
2. HTTP和HTTPS相关问题。
|
||||
|
||||
> 建议阅读阮一峰的[《HTTP 协议入门》](http://www.ruanyifeng.com/blog/2016/08/http.html)和[《SSL/TLS协议运行机制的概述》](http://www.ruanyifeng.com/blog/2014/02/ssl_tls.html)。
|
||||
|
||||
3. Linux常用命令和服务。
|
||||
|
||||
4. 进程和线程之间的关系。什么时候用多线程?什么时候用多进程?。
|
||||
|
||||
5. 关系型数据库相关问题(ACID、事务隔离级别、锁、SQL优化)。
|
||||
|
||||
6. 非关系型数据库相关问题(CAP/BASE、应用场景)。
|
||||
|
||||
### Python基础
|
||||
|
||||
1. 开发中用过哪些标准库和三方库。
|
||||
|
||||
> 标准库:sys / os / re / math / random / logging / json / pickle / shelve / socket / datetime / hashlib / configparser / urllib / itertools / collections / functools / threading / multiprocess / timeit / atexit / abc / asyncio / base64 / concurrent.futures / copy / csv / operator / enum / heapq / http / profile / pstats / ssl / unitest / uuid
|
||||
|
||||
2. 装饰器的作用、原理和实现。
|
||||
|
||||
3. 使用过哪些魔法方法。
|
||||
|
||||
> 建议阅读[《Python魔术方法指南》](https://pycoders-weekly-chinese.readthedocs.io/en/latest/issue6/a-guide-to-pythons-magic-methods.html)。
|
||||
|
||||
4. 生成式、生成器、迭代器的编写。
|
||||
|
||||
5. 列表、集合、字典的底层实现。
|
||||
|
||||
6. 垃圾回收相关问题。
|
||||
|
||||
7. 并发编程的相关问题。
|
||||
|
||||
8. 协程和异步I/O相关知识。
|
||||
|
||||
### Django和Flask
|
||||
|
||||
1. MVC架构(MTV)解决了什么问题。
|
||||
|
||||
2. 中间件的执行流程以及如何自定义中间件。
|
||||
|
||||
3. REST数据接口如何设计(URL、域名、版本、过滤、状态码、安全性)。
|
||||
|
||||
> 建议阅读阮一峰的[《RESTful API设计指南》](http://www.ruanyifeng.com/blog/2014/05/restful_api.html)。
|
||||
|
||||
4. 使用ORM框架实现CRUD操作的相关问题。
|
||||
|
||||
- 如何实现多条件组合查询 / 如何执行原生的SQL / 如何避免N+1查询问题
|
||||
|
||||
5. 如何执行异步任务和定时任务。
|
||||
|
||||
6. 如何实现页面缓存和查询缓存?缓存如何预热?
|
||||
|
||||
### 爬虫相关
|
||||
|
||||
1. Scrapy框架的组件和数据处理流程。
|
||||
2. 爬取的目的(项目中哪些地方需要用到爬虫的数据)。
|
||||
3. 使用的工具(抓包、下载、清理、存储、分析、可视化)。
|
||||
4. 数据的来源(能够轻松的列举出10个网站)。
|
||||
5. 数据的构成(抓取的某个字段在项目中有什么用)。
|
||||
6. 反反爬措施(限速、请求头、Cookie池、代理池、Selenium、PhantomJS、RoboBrowser、TOR、OCR)。
|
||||
7. 数据的体量(最后抓取了多少数据,多少W条数据或多少个G的数据)。
|
||||
8. 后期数据处理(持久化、数据补全、归一化、格式化、转存、分类)。
|
||||
|
||||
### 数据分析
|
||||
|
||||
1. 科学运算函数库(SciPy和NumPy常用运算)。
|
||||
2. 数据分析库(Pandas中封装的常用算法)。
|
||||
3. 常用的模型及对应的场景(分类、回归、聚类)。
|
||||
4. 提取了哪些具体的指标。
|
||||
5. 如何评价模型的优劣。
|
||||
6. 每种模型实际操作的步骤,对结果如何评价。
|
||||
|
||||
### 项目相关
|
||||
|
||||
1. 项目团队构成以及自己在团队中扮演的角色(在项目中的职责)。
|
||||
2. 项目的业务架构(哪些模块及子模块)和技术架构(移动端、PC端、后端技术栈)。
|
||||
3. 软件控制管理相关工具(版本控制、问题管理、持续集成)。
|
||||
4. 核心业务实体及其属性,实体与实体之间的关系。
|
||||
5. 用到哪些依赖库,依赖库主要解决哪方面的问题。
|
||||
6. 项目如何部署上线以及项目的物理架构(Nginx、Gunicorn/uWSGI、Redis、MongoDB、MySQL、Supervisor等)。
|
||||
7. 如何对项目进行测试,有没有做过性能调优。
|
||||
8. 项目中遇到的困难有哪些,如何解决的。
|
|
@ -1,25 +1,127 @@
|
|||
## 项目部署上线指南
|
||||
|
||||
### 更新Python环境到3.x
|
||||
### 准备上线
|
||||
|
||||
```Shell
|
||||
yum -y install gcc zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gdbm-devel db4-devel libpcap-devel xz-devel libffi-devel
|
||||
wget https://www.python.org/ftp/python/3.7.0/Python-3.7.0.tar.xz
|
||||
xz -d Python-3.7.0.tar.xz
|
||||
tar -xvf Python-3.7.0.tar
|
||||
cd Python-3.7.0
|
||||
./configure --prefix=/usr/local/python3 --enable-optimizations
|
||||
make && make install
|
||||
cd ~
|
||||
vim .bash_profile
|
||||
export PATH=$PATH:/usr/local/python3/bin
|
||||
ln -s /usr/local/python3/bin/python3 /usr/bin/python3
|
||||
ln -s /usr/local/python3/bin/pip3 /usr/bin/pip3
|
||||
```
|
||||
1. 上线前的检查工作。
|
||||
|
||||
```Shell
|
||||
python manage.py check --deploy
|
||||
```
|
||||
|
||||
2. 将DEBUG设置为False并配置ALLOWED_HOSTS。
|
||||
|
||||
```Python
|
||||
DEBUG = False
|
||||
ALLOWED_HOSTS = ['*']
|
||||
```
|
||||
|
||||
3. 安全相关的配置。
|
||||
|
||||
```Python
|
||||
# 保持HTTPS连接的时间
|
||||
SECURE_HSTS_SECONDS = 3600
|
||||
SECURE_HSTS_INCLUDE_SUBDOMAINS = True
|
||||
SECURE_HSTS_PRELOAD = True
|
||||
|
||||
# 自动重定向到安全连接
|
||||
SECURE_SSL_REDIRECT = True
|
||||
|
||||
# 避免浏览器自作聪明推断内容类型
|
||||
SECURE_CONTENT_TYPE_NOSNIFF = True
|
||||
|
||||
# 避免跨站脚本攻击
|
||||
SECURE_BROWSER_XSS_FILTER = True
|
||||
|
||||
# COOKIE只能通过HTTPS进行传输
|
||||
SESSION_COOKIE_SECURE = True
|
||||
CSRF_COOKIE_SECURE = True
|
||||
|
||||
# 防止点击劫持攻击手段 - 修改HTTP协议响应头
|
||||
# 当前网站是不允许使用<iframe>标签进行加载的
|
||||
X_FRAME_OPTIONS = 'DENY'
|
||||
```
|
||||
|
||||
4. 敏感信息放到环境变量或文件中。
|
||||
|
||||
```Python
|
||||
SECRET_KEY = os.environ['SECRET_KEY']
|
||||
|
||||
DB_USER = os.environ['DB_USER']
|
||||
DB_PASS = os.environ['DB_PASS']
|
||||
|
||||
REDIS_AUTH = os.environ['REDIS_AUTH']
|
||||
```
|
||||
|
||||
### 更新服务器Python环境到3.x
|
||||
|
||||
1. 安装底层依赖库。
|
||||
|
||||
```Shell
|
||||
yum -y install wget gcc zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gdbm-devel db4-devel libpcap-devel xz-devel libffi-devel
|
||||
```
|
||||
|
||||
2. 下载Python源代码。
|
||||
|
||||
```Shell
|
||||
wget https://www.python.org/ftp/python/3.7.0/Python-3.7.0.tar.xz
|
||||
```
|
||||
|
||||
3. 解压缩和解归档。
|
||||
|
||||
```Shell
|
||||
xz -d Python-3.7.0.tar.xz
|
||||
tar -xvf Python-3.7.0.tar
|
||||
```
|
||||
|
||||
4. 执行配置生成Makefile(构建文件)。
|
||||
|
||||
```Shell
|
||||
cd Python-3.7.0
|
||||
./configure --prefix=/usr/local/python37 --enable-optimizations
|
||||
```
|
||||
|
||||
5. 构建和安装。
|
||||
|
||||
```Shell
|
||||
make && make install
|
||||
```
|
||||
|
||||
6. 配置PATH环境变量并激活。
|
||||
|
||||
```Shell
|
||||
cd ~
|
||||
vim .bash_profile
|
||||
```
|
||||
|
||||
```INI
|
||||
... 此处省略上面的代码...
|
||||
|
||||
export PATH=$PATH:/usr/local/python37/bin
|
||||
|
||||
... 此处省略下面的代码...
|
||||
```
|
||||
|
||||
```Shell
|
||||
source .bash_profile
|
||||
```
|
||||
|
||||
7. 注册软链接(符号链接)。
|
||||
|
||||
```Shell
|
||||
ln -s /usr/local/python37/bin/python3.7 /usr/bin/python3
|
||||
ln -s /usr/local/python37/bin/pip3.7 /usr/bin/pip3
|
||||
```
|
||||
|
||||
8. 测试Python环境是否更新成功。
|
||||
|
||||
```Shell
|
||||
python3 --version
|
||||
python --version
|
||||
```
|
||||
|
||||
### 项目目录结构
|
||||
|
||||
下面是项目的目录结构,四个文件夹`conf`、`logs`、`src`和`venv`分别用来保存项目的配置文件、日志文件、源代码和虚拟环境。`conf`目录下的子目录`cert`中保存了配置HTTPS需要使用的证书和密钥。
|
||||
假设项目文件夹为`project`,下面的四个子目录分别是:`conf`、`logs`、`src`和`venv`分别用来保存项目的配置文件、日志文件、源代码和虚拟环境。其中,`conf`目录下的子目录`cert`中保存了配置HTTPS需要使用的证书和密钥;`src`目录下的项目代码可以通过版本控制工具从代码仓库中检出;虚拟环境可以通过venv或其他工具进行创建。
|
||||
|
||||
```
|
||||
project
|
||||
|
@ -35,13 +137,14 @@ project
|
|||
│ └── uwsgi.log
|
||||
├── requirements.txt
|
||||
├── src
|
||||
│ └── fang
|
||||
│ └── fangall
|
||||
│ ├── api
|
||||
│ ├── common
|
||||
│ ├── fang
|
||||
│ ├── forum
|
||||
│ ├── rent
|
||||
│ ├── user
|
||||
│ ├── manage.py
|
||||
│ ├── README.md
|
||||
│ ├── rent
|
||||
│ ├── static
|
||||
│ └── templates
|
||||
│
|
||||
|
@ -84,129 +187,188 @@ project
|
|||
└── pyvenv.cfg
|
||||
```
|
||||
|
||||
下面以阿里云为例,简单说明如何为项目注册域名、解析域名以及购买权威机构颁发的证书。
|
||||
|
||||
1. [注册域名](https://wanwang.aliyun.com/domain/)。
|
||||
|
||||
![](./res/aliyun-domain.png)
|
||||
|
||||
2. [域名备案](https://beian.aliyun.com/)。
|
||||
|
||||
![](./res/aliyun-keeprecord.png)
|
||||
|
||||
3. [域名解析](https://dns.console.aliyun.com/#/dns/domainList)。
|
||||
|
||||
![](./res/aliyun-dnslist.png)
|
||||
|
||||
![](./res/aliyun-resolve-settings.png)
|
||||
|
||||
4. [购买证书](https://www.aliyun.com/product/cas)。
|
||||
|
||||
![](./res/aliyun-certificate.png)
|
||||
|
||||
### uWSGI的配置
|
||||
|
||||
可以激活项目的虚拟环境并通过pip安装uWSGI。
|
||||
1. 在`project`目录下创建并激活虚拟环境。
|
||||
|
||||
```Shell
|
||||
pip install uwsgi
|
||||
```
|
||||
```Shell
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate
|
||||
```
|
||||
|
||||
`/root/project/conf/uwsgi.ini`
|
||||
2. 安装项目依赖项。
|
||||
|
||||
```INI
|
||||
[uwsgi]
|
||||
# 配置前导路径
|
||||
base=/root/project
|
||||
# 配置项目名称
|
||||
name=fang
|
||||
# 守护进程
|
||||
master=true
|
||||
# 进程个数
|
||||
processes=4
|
||||
# 虚拟环境
|
||||
pythonhome=%(base)/venv
|
||||
# 项目地址
|
||||
chdir=%(base)/src/%(name)
|
||||
# 指定python解释器
|
||||
pythonpath=%(pythonhome)/bin/python
|
||||
# 指定uwsgi文件
|
||||
module=%(name).wsgi
|
||||
# 通信的地址和端口(自己服务器的IP地址和端口)
|
||||
socket=172.18.61.250:8000
|
||||
# 日志文件地址
|
||||
logto = %(base)/logs/uwsgi.log
|
||||
```
|
||||
```Shell
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
可以先将“通信的地址和端口”项等号前面改为http来进行测试,如果没有问题再改回成socket,然后通过Nginx来实现项目的“动静分离”(静态资源交给Nginx处理,动态内容交给uWSGI处理)。
|
||||
3. 通过pip安装uWSGI。
|
||||
|
||||
```Shell
|
||||
uwsgi --ini uwsgi.ini &
|
||||
```
|
||||
```Shell
|
||||
pip install uwsgi
|
||||
```
|
||||
|
||||
4. 修改uWSGI的配置文件(`/root/project/conf/uwsgi.ini`)。
|
||||
|
||||
```INI
|
||||
[uwsgi]
|
||||
# 配置前导路径
|
||||
base=/root/project
|
||||
# 配置项目名称
|
||||
name=fangall
|
||||
# 守护进程
|
||||
master=true
|
||||
# 进程个数
|
||||
processes=4
|
||||
# 虚拟环境
|
||||
pythonhome=%(base)/venv
|
||||
# 项目地址
|
||||
chdir=%(base)/src/%(name)
|
||||
# 指定python解释器
|
||||
pythonpath=%(pythonhome)/bin/python
|
||||
# 指定uwsgi文件
|
||||
module=%(name).wsgi
|
||||
# 通信的地址和端口(自己服务器的IP地址和端口)
|
||||
socket=172.18.61.250:8000
|
||||
# 日志文件地址
|
||||
logto = %(base)/logs/uwsgi.log
|
||||
```
|
||||
|
||||
> 说明:可以先将“通信的地址和端口”项等号前面改为http来进行测试,如果没有问题再改回 成socket,然后通过Nginx来实现项目的“动静分离”(静态资源交给Nginx处理,动态内容交给 uWSGI处理)。按照下面的方式可以启动uWSGI服务器。
|
||||
|
||||
5. 启动服务器。
|
||||
|
||||
```Shell
|
||||
uwsgi --ini uwsgi.ini &
|
||||
```
|
||||
|
||||
### Nginx的配置
|
||||
#### 全局配置
|
||||
`/etc/nginx/nginx.conf`
|
||||
|
||||
```Nginx
|
||||
# 全局配置
|
||||
# 用户(可以设置为)
|
||||
user root;
|
||||
# 工作进程数(建议跟CPU的核数量一致)
|
||||
worker_processes auto;
|
||||
# 错误日志
|
||||
error_log /var/log/nginx/error.log;
|
||||
# 进程文件
|
||||
pid /run/nginx.pid;
|
||||
1. 修改全局配置文件(`/etc/nginx/nginx.conf`)。
|
||||
|
||||
# 包含其他的配置
|
||||
include /usr/share/nginx/modules/*.conf;
|
||||
```Nginx
|
||||
# 配置用户
|
||||
user root;
|
||||
# 工作进程数(建议跟CPU的核数量一致)
|
||||
worker_processes auto;
|
||||
# 错误日志
|
||||
error_log /var/log/nginx/error.log;
|
||||
# 进程文件
|
||||
pid /run/nginx.pid;
|
||||
# 包含其他的配置
|
||||
include /usr/share/nginx/modules/*.conf;
|
||||
# 工作模式和连接上限
|
||||
events {
|
||||
use epoll;
|
||||
worker_connections 1024;
|
||||
}
|
||||
# HTTP服务器相关配置
|
||||
http {
|
||||
# 日志格式
|
||||
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
|
||||
'$status $body_bytes_sent "$http_referer" '
|
||||
'"$http_user_agent" "$http_x_forwarded_for"';
|
||||
# 访问日志
|
||||
access_log /var/log/nginx/access.log main;
|
||||
# 开启高效文件传输模式
|
||||
sendfile on;
|
||||
# 用sendfile传输文件时有利于改善性能
|
||||
tcp_nopush on;
|
||||
# 禁用Nagle来解决交互性问题
|
||||
tcp_nodelay on;
|
||||
# 客户端保持连接时间
|
||||
keepalive_timeout 15;
|
||||
types_hash_max_size 2048;
|
||||
# 包含MIME类型的配置
|
||||
include /etc/nginx/mime.types;
|
||||
# 默认使用二进制流格式
|
||||
default_type application/octet-stream;
|
||||
# 包含其他配置文件
|
||||
include /etc/nginx/conf.d/*.conf;
|
||||
# 包含项目的Nginx配置文件
|
||||
include /root/project/conf/*.conf;
|
||||
}
|
||||
```
|
||||
|
||||
# 工作模式和连接上限
|
||||
events {
|
||||
use epoll;
|
||||
worker_connections 1024;
|
||||
}
|
||||
2. 编辑局部配置文件(`/root/project/conf/nginx.conf`)。
|
||||
|
||||
# HTTP服务器相关配置
|
||||
http {
|
||||
# 日志格式
|
||||
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
|
||||
'$status $body_bytes_sent "$http_referer" '
|
||||
'"$http_user_agent" "$http_x_forwarded_for"';
|
||||
# 访问日志
|
||||
access_log /var/log/nginx/access.log main;
|
||||
# 开启高效文件传输模式
|
||||
sendfile on;
|
||||
# 用sendfile传输文件时有利于改善性能
|
||||
tcp_nopush on;
|
||||
# 禁用Nagle来解决交互性问题
|
||||
tcp_nodelay on;
|
||||
# 客户端保持连接时间
|
||||
keepalive_timeout 15;
|
||||
types_hash_max_size 2048;
|
||||
# 包含MIME类型的配置
|
||||
include /etc/nginx/mime.types;
|
||||
# 默认使用二进制流格式
|
||||
default_type application/octet-stream;
|
||||
# 包含其他配置文件
|
||||
include /etc/nginx/conf.d/*.conf;
|
||||
```Nginx
|
||||
server {
|
||||
listen 80;
|
||||
server_name _;
|
||||
access_log /root/project/logs/access.log;
|
||||
error_log /root/project/logs/error.log;
|
||||
location / {
|
||||
include uwsgi_params;
|
||||
uwsgi_pass 172.18.61.250:8000;
|
||||
}
|
||||
location /static/ {
|
||||
alias /root/project/src/fangall/static/;
|
||||
expires 30d;
|
||||
}
|
||||
}
|
||||
server {
|
||||
listen 443;
|
||||
server_name _;
|
||||
ssl on;
|
||||
access_log /root/project/logs/access.log;
|
||||
error_log /root/project/logs/error.log;
|
||||
ssl_certificate /root/project/conf/cert/214915882850706.pem;
|
||||
ssl_certificate_key /root/project/conf/cert/214915882850706.key;
|
||||
ssl_session_timeout 5m;
|
||||
ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE:ECDH:AES:HIGH:!NULL:!aNULL:!MD5:!ADH:!RC4;
|
||||
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
|
||||
ssl_prefer_server_ciphers on;
|
||||
location / {
|
||||
include uwsgi_params;
|
||||
uwsgi_pass 172.18.61.250:8000;
|
||||
}
|
||||
location /static/ {
|
||||
alias /root/project/src/fangall/static/;
|
||||
expires 30d;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
# 包含项目的Nginx配置文件
|
||||
include /root/project/conf/*.conf;
|
||||
}
|
||||
```
|
||||
到此为止,我们可以启动Nginx来访问我们的应用程序,HTTP和HTTPS都是没有问题的,如果Nginx已经运行,在修改配置文件后,我们可以用下面的命令重新启动Nginx。
|
||||
|
||||
#### 局部配置
|
||||
3. 重启Nginx服务器。
|
||||
|
||||
`/root/project/conf/nginx.conf`
|
||||
```Shell
|
||||
nginx -s reload
|
||||
```
|
||||
|
||||
```Nginx
|
||||
server {
|
||||
listen 80;
|
||||
server_name _;
|
||||
access_log /root/project/logs/access.log;
|
||||
error_log /root/project/logs/error.log;
|
||||
location / {
|
||||
include uwsgi_params;
|
||||
uwsgi_pass 172.18.61.250:8000;
|
||||
}
|
||||
location /static/ {
|
||||
alias /root/project/src/fang/static/;
|
||||
expires 30d;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
到此为止,我们可以启动Nginx来访问我们的应用程序,HTTP和HTTPS都是没有问题的,如果Nginx已经运行,在修改配置文件后,我们可以用下面的命令重新启动Nginx。
|
||||
|
||||
```Shell
|
||||
nginx -s reload
|
||||
```
|
||||
> 说明:可以对Django项目使用`python manage.py collectstatic`命令将静态资源收集到指定目录下,要做到这点只需要在项目的配置文件`settings.py`中添加`STATIC_ROOT`配置即可。
|
||||
|
||||
#### 负载均衡配置
|
||||
|
||||
下面的配置中我们使用Nginx为HTTP、HTTPS以及Redis配置负载均衡。
|
||||
下面的配置中我们使用Nginx实现负载均衡,为另外的三个Nginx服务器(通过Docker创建)提供反向代理服务。
|
||||
|
||||
```Shell
|
||||
docker run -d -p 801:80 --name nginx1 nginx:latest
|
||||
docker run -d -p 802:80 --name nginx2 nginx:latest
|
||||
docker run -d -p 803:80 --name nginx3 nginx:latest
|
||||
```
|
||||
|
||||
```Nginx
|
||||
user root;
|
||||
|
@ -220,24 +382,12 @@ events {
|
|||
worker_connections 1024;
|
||||
}
|
||||
|
||||
stream {
|
||||
upstream redis.local {
|
||||
server 172.18.61.250:36379;
|
||||
server 172.18.61.250:46379;
|
||||
server 172.18.61.250:56379;
|
||||
}
|
||||
|
||||
server {
|
||||
listen 6379;
|
||||
proxy_pass redis.local;
|
||||
}
|
||||
}
|
||||
|
||||
# 为HTTP服务配置负载均衡
|
||||
http {
|
||||
upstream fang.com {
|
||||
server 172.18.61.250:801;
|
||||
server 172.18.61.250:802;
|
||||
server 172.18.61.250:803;
|
||||
server 172.18.61.250:801 weight=4;
|
||||
server 172.18.61.250:802 weight=2;
|
||||
server 172.18.61.250:803 weight=2;
|
||||
}
|
||||
|
||||
server {
|
||||
|
@ -267,12 +417,193 @@ http {
|
|||
}
|
||||
```
|
||||
|
||||
> 说明:上面的配置文件中的Nginx服务器(3个节点)和Redis服务器(2个节点,每个节点是1个master和2个slave的配置)都是通过Docker来创建的,实际部署的时候无论是否使用Docker进行部署,这些主机应该都是独立的服务器。
|
||||
> 说明:Nginx在配置负载均衡时,默认使用WRR(加权轮询算法),除此之外还支持ip_hash、fair(需要安装upstream_fair模块)和url_hash算法。此外,在配置upstream模块时可以指定服务器的状态值,包括:backup(备份机器,其他服务器不可用时才将请求分配到该机器)、down、fail_timeout(请求失败达到max_fails后的暂停服务时间)、max_fails(允许请求失败的次数)和weight(轮询的权重)。
|
||||
|
||||
### Keepalived
|
||||
|
||||
当使用Nginx进行负载均衡配置时,要考虑负载均衡服务器宕机的情况。为此可以使用Keepalived来实现负载均衡主机和备机的热切换,从而保证系统的高可用性。Keepalived的配置还是比较复杂,通常由专门做运维的人进行配置,一个基本的配置可以参照[《Keepalived的配置和使用》](https://www.jianshu.com/p/dd93bc6d45f5)。
|
||||
|
||||
### MySQL主从复制
|
||||
|
||||
下面还是基于Docker来演示如何配置MySQL主从复制。我们事先准备好MySQL的配置文件以及保存MySQL数据和运行日志的目录,然后通过Docker的数据卷映射来指定容器的配置、数据和日志文件的位置。
|
||||
|
||||
```Shell
|
||||
root
|
||||
└── mysql
|
||||
├── conf
|
||||
│ ├── master
|
||||
│ │ └── mysqld.cnf
|
||||
│ ├── slave1
|
||||
│ │ └── mysqld.cnf
|
||||
│ ├── slave2
|
||||
│ │ └── mysqld.cnf
|
||||
│ └── slave3
|
||||
│ └── mysqld.cnf
|
||||
└── data
|
||||
├── master
|
||||
├── slave1
|
||||
├── slave2
|
||||
└── slave3
|
||||
```
|
||||
|
||||
1. MySQL的配置文件(master和slave的配置文件需要不同的server-id)。
|
||||
|
||||
```
|
||||
[mysqld]
|
||||
pid-file=/var/run/mysqld/mysqld.pid
|
||||
socket=/var/run/mysqld/mysqld.sock
|
||||
datadir=/var/lib/mysql
|
||||
log-error=/var/log/mysql/error.log
|
||||
server-id=1
|
||||
log_bin=/var/log/mysql/mysql-bin.log
|
||||
expire_logs_days=30
|
||||
max_binlog_size=256M
|
||||
symbolic-links=0
|
||||
```
|
||||
|
||||
2. 创建和配置master。
|
||||
|
||||
```Shell
|
||||
docker run -d -p 3306:3306 --name mysql57 \
|
||||
-v /root/mysql/conf/master:/etc/mysql/mysql.conf.d \
|
||||
-v /root/mysql/data/master:/var/lib/mysql \
|
||||
-e MYSQL_ROOT_PASSWORD=123456 mysql:5.7
|
||||
docker exec -it mysql57 /bin/bash
|
||||
```
|
||||
|
||||
```Shell
|
||||
mysql -u root -p
|
||||
Enter password:
|
||||
Welcome to the MySQL monitor. Commands end with ; or \g.
|
||||
Your MySQL connection id is 1
|
||||
Server version: 5.7.23-log MySQL Community Server (GPL)
|
||||
Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.
|
||||
Oracle is a registered trademark of Oracle Corporation and/or its
|
||||
affiliates. Other names may be trademarks of their respective
|
||||
owners.
|
||||
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
|
||||
|
||||
mysql> grant replication slave on *.* to 'slave'@'%' identified by 'iamslave';
|
||||
Query OK, 0 rows affected, 1 warning (0.00 sec)
|
||||
|
||||
mysql> flush privileges;
|
||||
Query OK, 0 rows affected (0.00 sec)
|
||||
|
||||
mysql> show master status;
|
||||
+------------------+----------+--------------+------------------+-------------------+
|
||||
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
|
||||
+------------------+----------+--------------+------------------+-------------------+
|
||||
| mysql-bin.000001 | 590 | | | |
|
||||
+------------------+----------+--------------+------------------+-------------------+
|
||||
1 row in set (0.00 sec)
|
||||
|
||||
mysql> quit
|
||||
Bye
|
||||
exit
|
||||
```
|
||||
|
||||
上面创建Docker容器时使用的`-v`参数(`--volume`)表示映射数据卷,冒号前是宿主机的目录,冒号后是容器中的目录,这样相当于将宿主机中的目录挂载到了容器中。
|
||||
|
||||
3. 创建和配置slave。
|
||||
|
||||
```Shell
|
||||
docker run -d -p 3307:3306 --name mysql57-slave-1 \
|
||||
-v /root/mysql/conf/slave1:/etc/mysql/mysql.conf.d \
|
||||
-v /root/mysql/data/slave1:/var/lib/mysql \
|
||||
-e MYSQL_ROOT_PASSWORD=123456 \
|
||||
--link mysql57:mysql57 mysql:5.7
|
||||
docker exec -it mysql57-slave-1 /bin/bash
|
||||
```
|
||||
|
||||
```Shell
|
||||
mysql -u root -p
|
||||
Enter password:
|
||||
Welcome to the MySQL monitor. Commands end with ; or \g.
|
||||
Your MySQL connection id is 2
|
||||
Server version: 5.7.23-log MySQL Community Server (GPL)
|
||||
Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.
|
||||
Oracle is a registered trademark of Oracle Corporation and/or its
|
||||
affiliates. Other names may be trademarks of their respective
|
||||
owners.
|
||||
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
|
||||
|
||||
mysql> reset slave;
|
||||
Query OK, 0 rows affected (0.02 sec)
|
||||
|
||||
mysql> change master to master_host='mysql57', master_user='slave', master_password='iamslave', master_log_file='mysql-bin.000003', master_log_pos=590;
|
||||
Query OK, 0 rows affected, 2 warnings (0.03 sec)
|
||||
|
||||
mysql> start slave;
|
||||
Query OK, 0 rows affected (0.01 sec)
|
||||
|
||||
mysql> show slave status\G
|
||||
*************************** 1. row ***************************
|
||||
Slave_IO_State: Waiting for master to send event
|
||||
Master_Host: mysql57
|
||||
Master_User: slave
|
||||
Master_Port: 3306
|
||||
Connect_Retry: 60
|
||||
Master_Log_File: mysql-bin.000001
|
||||
Read_Master_Log_Pos: 590
|
||||
Relay_Log_File: f352f05eb9d0-relay-bin.000002
|
||||
Relay_Log_Pos: 320
|
||||
Relay_Master_Log_File: mysql-bin.000001
|
||||
Slave_IO_Running: Yes
|
||||
Slave_SQL_Running: Yes
|
||||
Replicate_Do_DB:
|
||||
Replicate_Ignore_DB:
|
||||
Replicate_Do_Table:
|
||||
Replicate_Ignore_Table:
|
||||
Replicate_Wild_Do_Table:
|
||||
Replicate_Wild_Ignore_Table:
|
||||
Last_Errno: 0
|
||||
Last_Error:
|
||||
Skip_Counter: 0
|
||||
Exec_Master_Log_Pos: 590
|
||||
Relay_Log_Space: 534
|
||||
Until_Condition: None
|
||||
Until_Log_File:
|
||||
Until_Log_Pos: 0
|
||||
Master_SSL_Allowed: No
|
||||
Master_SSL_CA_File:
|
||||
Master_SSL_CA_Path:
|
||||
Master_SSL_Cert:
|
||||
Master_SSL_Cipher:
|
||||
Master_SSL_Key:
|
||||
Seconds_Behind_Master: 0
|
||||
Master_SSL_Verify_Server_Cert: No
|
||||
Last_IO_Errno: 0
|
||||
Last_IO_Error:
|
||||
Last_SQL_Errno: 0
|
||||
Last_SQL_Error:
|
||||
Replicate_Ignore_Server_Ids:
|
||||
Master_Server_Id: 1
|
||||
Master_UUID: 30c38043-ada1-11e8-8fa1-0242ac110002
|
||||
Master_Info_File: /var/lib/mysql/master.info
|
||||
SQL_Delay: 0
|
||||
SQL_Remaining_Delay: NULL
|
||||
Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
|
||||
Master_Retry_Count: 86400
|
||||
Master_Bind:
|
||||
Last_IO_Error_Timestamp:
|
||||
Last_SQL_Error_Timestamp:
|
||||
Master_SSL_Crl:
|
||||
Master_SSL_Crlpath:
|
||||
Retrieved_Gtid_Set:
|
||||
Executed_Gtid_Set:
|
||||
Auto_Position: 0
|
||||
Replicate_Rewrite_DB:
|
||||
Channel_Name:
|
||||
Master_TLS_Version:
|
||||
1 row in set (0.00 sec)
|
||||
|
||||
mysql> quit
|
||||
Bye
|
||||
exit
|
||||
```
|
||||
|
||||
接下来可以如法炮制配置出slave2和slave3,这样就可以搭建起一个“一主带三从”的主从复制环境。上面创建创建容器时使用的`--link`参数用来配置容器在网络上的主机名(网络地址别名),下一节有这个知识点的介绍。
|
||||
|
||||
### Docker
|
||||
|
||||
事实上,项目上线中最为麻烦的事情就是配置软件运行环境,环境的差异会给软件的安装和部署带来诸多的麻烦,而Docker正好可以解决这个问题。关于Docker在之前的文档中我们已经介绍过了,接下来我们对Docker的知识做一些必要的补充。
|
||||
|
@ -334,6 +665,81 @@ http {
|
|||
```Shell
|
||||
docker run --link <container-name>:<alias-name>
|
||||
```
|
||||
|
||||
|
||||
我们在Docker中完成项目的部署,并且将整个部署好的容器打包成镜像文件进行分发和安装,这样就可以解决项目在多个节点上进行部署时可能遇到的麻烦。
|
||||
|
||||
如果我们能够在Docker中完成项目的部署,并且将整个部署好的容器打包成镜像文件进行分发和安装,这样就可以解决项目在多个节点上进行部署时可能遇到的麻烦,而且整个部署可以在很短的时间内完成。
|
||||
|
||||
### Supervisor
|
||||
|
||||
[Supervisor](https://github.com/Supervisor/supervisor)是一个用Python写的进程管理工具,可以很方便的用来在类Unix系统下启动、重启(自动重启程序)和关闭进程。
|
||||
|
||||
1. 安装Supervisor。
|
||||
|
||||
```Shell
|
||||
yum -y install supervisor
|
||||
```
|
||||
|
||||
2. 查看Supervisor的配置文件。
|
||||
|
||||
```Shell
|
||||
vim /etc/supervisord.conf
|
||||
...
|
||||
[include]
|
||||
files = supervisord.d/*.ini
|
||||
```
|
||||
|
||||
可以看出自定义的管理配置代码可以放在`/etc/supervisord.d`目录中,并且文件名以`ini`作为后缀即可。
|
||||
|
||||
3. 编写管理配置代码。
|
||||
|
||||
```Shell
|
||||
cd /etc/supervisord.d
|
||||
vim fang.ini
|
||||
```
|
||||
|
||||
```INI
|
||||
[program:fang]
|
||||
...
|
||||
```
|
||||
|
||||
4. 启动Supervisor服务和查看状态。
|
||||
|
||||
```Shell
|
||||
systemctl start supervisord
|
||||
supervisorctl status
|
||||
```
|
||||
|
||||
### 其他服务
|
||||
|
||||
1. 常用开源软件。
|
||||
|
||||
| 功能 | 开源方案 |
|
||||
| ------------------- | ------------------------- |
|
||||
| 版本控制工具 | Git、Mercurial、SVN |
|
||||
| 缺陷管理 | Redmine、Mantis |
|
||||
| 负载均衡 | Nginx、LVS、HAProxy |
|
||||
| 邮件服务 | Postfix、Sendmail |
|
||||
| HTTP服务 | Nginx、Apache |
|
||||
| 消息队列 | RabbitMQ、ZeroMQ、Redis |
|
||||
| 文件系统 | FastDFS |
|
||||
| 基于位置服务(LBS) | MongoDB、Redis |
|
||||
| 监控服务 | Nagios、Zabbix |
|
||||
| 关系型数据库 | MySQL、PostgreSQL |
|
||||
| 非关系型数据库 | MongoDB、Redis、Cassandra |
|
||||
| 搜索引擎 | ElasticSearch、Solr |
|
||||
| 缓存服务 | Mamcached、Redis |
|
||||
|
||||
2. 常用云服务。
|
||||
|
||||
| 功能 | 可用的云服务 |
|
||||
| -------------- | --------------------------------------- |
|
||||
| 团队协作工具 | Teambition、钉钉 |
|
||||
| 代码托管平台 | Github、Gitee、CODING |
|
||||
| 邮件服务 | SendCloud |
|
||||
| 云存储(CDN) | 七牛、OSS、LeanCloud、Bmob、又拍云、AWS |
|
||||
| 移动端推送 | 极光、友盟、百度 |
|
||||
| 即时通信 | 环信、融云 |
|
||||
| 短信服务 | 云片、极光、Luosimao、又拍云 |
|
||||
| 第三方登录 | 友盟、ShareSDK |
|
||||
| 网站监控和统计 | 阿里云监控、监控宝、百度云观测、小鸟云 |
|
||||
|
||||
|
|