关于
Git HTTP 服务器是代码托管服务最重要的组件之一,Git HTTP 服务器将 HTTP 请求的数据写入到 git-upload-pack/git-receive-pack 的标准输入,然后读取 git-upload-pack/git-receive-pack 的输出,写入 HTTP 响应包体,然后传输给客户端。原理非常简单。
DotNet 安装
首先需要下载 .NET Core ,此站点为正式释放版本,如果要体验新的版本可以去 Github 项目主页下载最新的:Github:
在 Windows 上,dotnet 提供安装程序,直接点击安装即可,如果你的操作系统在列表中,可以支持下载安装程序。
到目前(20160525)为止,dotnet 对 Ubuntu 16.04 的支持稍微有些问题,dotnet 依赖 libicu52 而 Ubuntu 16.04 的版本是 libicu55,并且 Ubuntu 的源里面没有 libicu52,解决方案是从 ubuntu 14.04 的源下载即可, 下载地址: dotnet 绝大部分功能还是能够正常运行的。
DotNet 命令
通常来说,dotnet 创建一个项目非常简单,进入到一个目录,运行
dotnet new
还原依赖
dotnet restore
编译
dotnet build
运行
dotnet run args...
发布
dotnet publish
项目依赖
对于 dotnet 项目而言,目前 RC2 为止,项目文件为 project.json (由于 project.json 的局限性,Microsoft 将迁移 project.json 到 csproj)
{ "title": "KStone GIT HTTP Server - Aquila", "copyright": "Copyright © 2016. Force Charlie. All Rights Reserved.", "description": "KStone GIT HTTP Server - Aquila ", "version": "1.0.0-*", "buildOptions": { "emitEntryPoint": true }, "dependencies": { "Microsoft.NETCore.App": { "type": "platform", "version": "1.0.0-rc2-3002702" }, "Microsoft.AspNetCore.Server.IISIntegration": "1.0.0-rc2-final", "Microsoft.AspNetCore.Server.Kestrel": "1.0.0-rc2-final", "Microsoft.AspNetCore.StaticFiles": "1.0.0-rc2-final", "Microsoft.AspNetCore.Diagnostics": "1.0.0-rc2-final", "Microsoft.Extensions.Configuration.Json": "1.0.0-rc2-final", "Microsoft.AspNetCore.Routing": "1.0.0-rc2-final", "Microsoft.Extensions.Options.ConfigurationExtensions": "1.0.0-rc2-final", "Microsoft.Extensions.Configuration.EnvironmentVariables": "1.0.0-rc2-final", "Microsoft.Extensions.Logging": "1.0.0-rc2-final", "Microsoft.Extensions.Logging.Console": "1.0.0-rc2-final", "Microsoft.Extensions.Logging.Debug": "1.0.0-rc2-final" }, "frameworks": { "netcoreapp1.0": { "imports": "dnxcore50" } }, "tools": { "Microsoft.AspNetCore.Server.IISIntegration.Tools": { "version": "1.0.0-preview1-final", "imports": "portable-net45+win8+dnxcore50" } }}
在实现 Git HTTP 服务器的过程中,我选择了 Kestrel 库,Kestrel 是 ASP.NET 团队基于 libuv 实现的一个高效的 HTTP Web Server 库。项目地址:
Kestrel 服务器入口
运行一个 HTTP 服务器,在 Kestrel 的眼中就是一个 WebHostBuilder,WebHostBuilder 绑定好参数就可以运行了,ASP.NET 团队做了很多事情,比如多线程,TCP 设置 NoDelay ,开启 HTTPS , 开启 HTTPS 要额外的证书,设置网站根目录,UseStartup 是一种模板类,实现的类必须拥有 Configure 方法。关于 Unix domain socket,笔者并未测试。
var host = new WebHostBuilder() .UseKestrel(options => { options.ThreadCount = 4; options.NoDelay = true; //options.UseHttps("testCert.pfx", "testPassword"); options.UseConnectionLogging(); }) .UseUrls("http://0.0.0.0:5000") .UseContentRoot(Directory.GetCurrentDirectory()) .UseStartup() .Build(); // The following section should be used to demo sockets //var addresses = application.GetAddresses(); //addresses.Clear(); //addresses.Add("http://unix:/tmp/kestrel-test.sock"); host.Run();
服务器配置与初始化
熟悉 .NET 的都知道,以前读取配置文件有 System.Configuration ,通过读取 XML 格式后缀名为 .config 的配置文件, ASP.NET 开源后,实现了对 JSON 格式配置文件的默认支持。
以 下面的配置文件(appsettings.json)为例:
{ "Logging": { "IncludeScopes": false, "LogLevel": { "Default": "Verbose", "System": "Information", "Microsoft": "Information" } }, "AquilaSettings": { "GitPath": "/usr/bin/git", "Repositories": "/home/git/repositories", "PathConvert":true }}
其中 Logging 是 Kestrel 服务器去解析,而 AquilaSettings 是用户自定义的解析,解析这一部分第一步是 定义一个类
public class AquilaSettings { public string GitPath { get; set; } public string Repositories { get; set; } public bool PathConvert { get; set; } }
然后在启动类中解析即可,也就是 ConfigureServices 方法中,这里利用了 JSON 的反序列化。启动类可以有 public Startup(IHostingEnvironment env) 这样的构造函数,以助于服务器获得 Host 环境。
public class AquilaContext { public IConfigurationRoot Configuration { get; set; } public AquilaContext(IHostingEnvironment env) { var builder = new ConfigurationBuilder() .SetBasePath(env.ContentRootPath) .AddJsonFile("appsettings.json") .AddEnvironmentVariables(); Configuration = builder.Build(); } public void ConfigureServices(IServiceCollection services) { services.Configure(Configuration.GetSection("AquilaSettings")); } public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory, IOptions aquilaSettings) { app.Run(async context => { //AquilaSession session=new AquilaSession(context,aquilaSettings.Value); await AquilaSession.Process(context, aquilaSettings.Value); //await context.Response.WriteAsync("Hello world"); }); } }
服务器请求与响应
在 Kestrel 的世界观中, HTTP 请求和响应由 HttpContext 抽象, HttpContext 包含由 Request, Response 以及其他诸多的 .NET 类的成员和方法。
GIT HTTP 服务器只需要实现 GET POST 两类请求,在本文中,我并不打算实现哑协议,只实现智能协议。 GIT 对远程仓库的操作本质上只有两种或三种, 分别是 fetch,push 以及 ssh 支持的 archive ,HTTP 服务器也就实现 fetch 和 push 即可,而 clone 本质上是 fetch 的一种特例,这里也就不做讨论。
主要功能在此类中已经实现:
public class AquilaSession { private HttpContext context; private AquilaSettings settings; ///NOT allowed dump protocol private bool ParsePassword(out string username, out string password) { username = string.Empty; password = string.Empty; if (!context.Request.Headers.ContainsKey("Authorization")) { return false; } var basicText=context.Request.Headers["Authorization"]; return true; } private void SkipAuthenticate() { context.Response.Headers["WWW-Authenticate"] = "Basic realm=\"\""; context.Response.StatusCode = StatusCodes.Status401Unauthorized; context.Response.WriteAsync("Unauthorized"); } private void TransmitRefs() { var url = context.Request.Path.ToString(); if (!url.EndsWith("/info/refs") || !context.Request.Query.ContainsKey("service")) { context.Response.StatusCode = StatusCodes.Status400BadRequest; context.Response.WriteAsync("Bad Request !"); return; } //var path=url.Substring(0,url.Length-"/info/refs".Length); var path = url.Substring(0, url.Length - 10); var repodir = AquilaPathCombine.PathCombine(settings.Repositories, path, settings.PathConvert); context.Response.Headers["Expires"] = "Fri, 01 Jan 1980 00:00:00 GMT"; context.Response.Headers["Pragma"] = "no-cache"; context.Response.Headers["Cache-Control"] = "no-cache, max-age=0, must-revalidate"; Process process = new Process(); process.StartInfo.FileName = settings.GitPath; process.StartInfo.RedirectStandardError = true; process.StartInfo.RedirectStandardOutput = true; var service = context.Request.Query["service"]; if (service == "git-upload-pack") { process.StartInfo.Arguments = "upload-pack --stateless-rpc --advertise-refs \"" + repodir + "\""; context.Response.ContentType = "application/x-git-upload-pack-advertisement"; var bytes = System.Text.Encoding.UTF8.GetBytes("001e# service=git-upload-pack\n0000"); context.Response.Body.Write(bytes, 0, bytes.Length); } else if (service == "git-receive-pack") { process.StartInfo.Arguments = "receive-pack --stateless-rpc --advertise-refs \"" + repodir + "\""; context.Response.ContentType = "application/x-git-receive-pack-advertisement"; var bytes = System.Text.Encoding.UTF8.GetBytes("001f# service=git-receive-pack\n0000"); context.Response.Body.Write(bytes, 0, bytes.Length); } else { context.Response.StatusCode = StatusCodes.Status400BadRequest; context.Response.WriteAsync("Invalid service !"); return; } process.Start(); if (process.StandardError.BaseStream.CanRead) { process.StandardError.BaseStream.CopyToAsync(context.Response.Body); } if (process.StandardOutput.BaseStream.CanRead) { process.StandardOutput.BaseStream.CopyToAsync(context.Response.Body); } process.WaitForExit(); } private void TransmitPackets() { var url = context.Request.Path.ToString(); var index = url.LastIndexOf('/'); if (index == -1) { context.Response.StatusCode = StatusCodes.Status400BadRequest; context.Response.WriteAsync("Bad Request !"); return; } var service = url.Substring(index + 1); var path = url.Substring(0, index); var repodir = AquilaPathCombine.PathCombine(settings.Repositories, path, settings.PathConvert); /// context.Response.Headers["Expires"] = "Fri, 01 Jan 1980 00:00:00 GMT"; context.Response.Headers["Pragma"] = "no-cache"; context.Response.Headers["Cache-Control"] = "no-cache, max-age=0, must-revalidate"; Process process = new Process(); process.StartInfo.FileName = settings.GitPath; process.StartInfo.RedirectStandardError = true; process.StartInfo.RedirectStandardOutput = true; process.StartInfo.RedirectStandardInput = true; if (service == "git-upload-pack") { process.StartInfo.Arguments = "upload-pack --stateless-rpc \"" + repodir + "\""; context.Response.ContentType = "application/x-git-upload-pack-result"; } else if (service == "git-receive-pack") { process.StartInfo.Arguments = "receive-pack --stateless-rpc \"" + repodir + "\""; context.Response.ContentType = "application/x-git-receive-pack-result"; } else { context.Response.StatusCode = StatusCodes.Status400BadRequest; context.Response.WriteAsync("Invalid service !"); return; } Stream input = null; process.Start(); if (context.Request.Headers.ContainsKey("Content-Encoding") && context.Request.Headers["Content-Encoding"].Equals("gzip")) { input = new GZipStream(context.Request.Body, CompressionMode.Decompress); } else { input = context.Request.Body; } if (process.StandardInput.BaseStream.CanWrite) { input.CopyToAsync(process.StandardInput.BaseStream); } if (process.StandardError.BaseStream.CanRead) { process.StandardError.BaseStream.CopyToAsync(context.Response.Body); } if (process.StandardOutput.BaseStream.CanRead) { process.StandardOutput.BaseStream.CopyToAsync(context.Response.Body); } process.WaitForExit(); } public AquilaSession(HttpContext context_, AquilaSettings settings_) { context = context_; settings = settings_; } public static async Task Process(HttpContext context_, AquilaSettings settings_) { AquilaSession session = new AquilaSession(context_, settings_); context_.Response.Headers["Server"] = "Aquila/1.0"; switch (context_.Request.Method) { case "GET": session.TransmitRefs(); break; case "POST": session.TransmitPackets(); break; default: { context_.Response.StatusCode = StatusCodes.Status405MethodNotAllowed; await context_.Response.WriteAsync("Method Not Allowed"); } break; } return 0; } }
在笔者使用 .NET Core 实现 GIT HTTP 服务器的过程中,觉得最方便的是 C# Stream 的 CopyToAsync ,将 Request Body 拷贝到 git 命令的标准输入,然后将 命令的输出和错误 拷贝到 Response 的 Body 中,一切就完成了,需要注意 要等待进程的退出,不然会有数据丢失,即调用 WaitForExit() 函数。
运行
在 Windows 10 上运行成功
在 Ubuntu 16.04 上运行成功关于源码
由于一些其他功能都没有实现,以及 .NET Core 1.0 RTM 还没释放,所以目前我也不打算放出来。